Français
Applied AI · Advanced 🔴 · Session 4
📝 Teacher's Guide
← Return to program 📄 Source .md

Applied AI — Advanced Level

Session 4: Multi-agent architecture

Program : Applied AI — Professional training in artificial intelligence Instructor: Yann Isola Level : Advanced — Solutions architects preparing for certification Claude Certified Architect Recommended duration: 3.5 hours (2 hours of lecture + 1.5 hours of practical exercises) Prerequisites: Sessions 1 to 3 of the advanced level (orchestration, tools/function calling, context management)

Educational objectives

At the end of this session, participants will be able to:

  1. Choose and justify a task decomposition strategy (vertical, horizontal, recursive) according to complexity, token budget and specialization benefit.
  2. Design a complete escalation policy with Human-in-the-Loop (HITL): triggers, escalation reasons, integration of human feedback and SLA management (Service Level Agreement — service level agreement).
  3. To implement error management patterns specific to multi-agent systems: idempotent retry, state checkpointing, circuit breaker (software circuit breaker), graceful degradation and dead letter queues (DLQ — Dead Letter Queue ).
  4. Instrument a multi-agent system with distributed tracing and correlation identifiers.

Session plan

Block Duration Content
1 40 mins Task Decomposition Strategies (Chapter 8)
2 40 mins Climbing and Human-in-the-Loop (Chapter 9)
3 40 mins Multi-agent error management (Chapter 10)
4 90 mins Practical exercises + interactive demonstration (web page)
5 10 mins Validation and summary quiz

Block 1 — Task decomposition strategies

1.1 Why decompose?

A single agent faced with a complex task encounters three walls:

Decomposition transforms a monolithic problem into a graph of subtasks entrusted to dedicated agents, coordinated by a orchestrator .

Educational point: emphasize that decomposition is not free. Each additional agent adds latency, token cost, and operational complexity. The question is never “can we decompose?” » but “does the gain exceed the cost?” ".

1.2 Vertical decomposition (by depth)

Principle: break down the workflow into sequential steps of different nature. The canonical pattern:

Planification → Exécution → Vérification

Benefits :

Boundaries :

Example implementation (Python pseudocode):

class VerticalPipeline:
    def __init__(self, planner, executor, verifier, max_repairs=2):
        self.planner = planner        # Agent LLM avec prompt "planificateur"
        self.executor = executor      # Agent LLM avec accès aux outils
        self.verifier = verifier      # Agent LLM avec prompt "auditeur"
        self.max_repairs = max_repairs

    def run(self, task: str) -> Result:
        plan = self.planner.plan(task)          # → liste d'étapes + critères
        result = self.executor.execute(plan)    # → artefacts produits
        for attempt in range(self.max_repairs):
            report = self.verifier.check(result, plan.success_criteria)
            if report.ok:
                return Result(status="success", artifacts=result)
            # Le rapport d'audit devient une consigne de réparation
            result = self.executor.repair(result, report.issues)
        return Result(status="needs_human_review", artifacts=result)

Note the output needs_human_review : vertical decomposition naturally articulates with escalation (Block 2).

1.3 Horizontal decomposition (by domain)

Principle: cut by area of ​​expertise . Each agent owns a domain and processes it end-to-end.

Example — analysis of a financial compliance file:

                    ┌──────────────────┐
                    │  Orchestrateur   │
                    └────────┬─────────┘
         ┌──────────────┬────┴─────────┬───────────────┐
         ▼              ▼              ▼               ▼
  Agent juridique  Agent fiscal   Agent risque   Agent rédaction
  (réglementation) (imposition)   (scoring)      (synthèse finale)

Benefits :

Boundaries :

1.4 Recursive decomposition

Principle: each agent can itself decompose its subtask and delegate to subagents, forming a delegation tree.

Orchestrateur racine
├── Agent A (sous-tâche 1)
│   ├── Sous-agent A1
│   └── Sous-agent A2
│       └── Sous-sous-agent A2a
└── Agent B (sous-tâche 2)

Essential safety rules:

  1. Maximum depth (max_depth ): without limit, an agent can create an infinite recursion of delegations. Typical value: 2 to 3 levels. ⚠ (indicative value, to be calibrated according to your load)
  2. Legacy token budget : each level receives a fraction of the parent budget. If the parent has 100,000 tokens, he or she only delegates, for example, 30,000 per child.
  3. Results contract : each subagent returns a structured format (JSON with status, artifacts, cost consumed), never unconstrained free text.

When recursion shines: problems with a fractal structure — code review of a monorepo (each module is broken down into files), documentary due diligence (each file is broken down into parts), site generation (each page is broken down into sections).

1.5 When to decompose? The three decision criteria

Criteria Question to ask Indicative threshold
Complexity threshold Does the task have more than N heterogeneous steps? Can a single system prompt cover all cases? > 5–7 distinct steps ⚠
Token budget Does the necessary context (documents + history + tools) exceed ~60–70% of the window? ⚠ Predictable saturation → break down
Specialization benefit Would specialized prompts measurably improve quality by domain? Quality gain > orchestration overhead

Decision tree to present on the board:

  1. Does the task fit comfortably in a single context with acceptable quality? → Don't break it down. (The most robust architecture is the one that doesn’t exist.)
  2. Are the steps natures different (plan vs. execute vs. check)? → Vertical.
  3. Do the subproblems fall under domains different and parallelizable? → Horizontal.
  4. Is the structure unpredictable or fractal? → Recursive , with strict safeguards.
  5. Real cases: often hybrid — a vertical pipeline whose execution stage is horizontal.

Certification pitfall: exam questions often oppose vertical and horizontal on an ambiguous case. The reliable discriminator: sequentiality of different natures → vertical; domain parallelism → horizontal.

Block 2 — Climbing and Human-in-the-Loop (HITL)

2.1 Why climbing is an architectural requirement, not an option

No agent system achieves 100% reliable autonomy. The architect must design from the start the paths by which the system gives back to humans. A system without explicit escalation still escalates — but in a chaotic way: support tickets, incidents, loss of confidence.

HITL (Human-in-the-Loop) designates the set of mechanisms by which a human validates, corrects or takes over the work of an agent.

2.2 The three escalation triggers

a) Confidence below the threshold

The agent (or an evaluator model) produces a confidence score on its own output. Below a defined threshold, the task is escalated.

CONFIDENCE_THRESHOLD = 0.75  # ⚠ à calibrer sur données réelles, jamais au doigt mouillé

assessment = judge_agent.evaluate(answer, context)
if assessment.confidence < CONFIDENCE_THRESHOLD:
    escalate(task, reason="low_confidence", score=assessment.confidence)

Critical point to teach: LLMs (Large Language Models — large language models) are poorly calibrated to their own self-reported confidence. Best practices:

b) Out-of-scope detection

The agent recognizes that the request falls outside its authorized domain: subject not covered, language not supported, request for regulated advice (legal, medical, financial), attempted circumvention.

Typical implementation: a perimeter classifier upstream (small, fast model) + an auto-detection instruction in the agent's prompt system (defense in depth — two layers are better than one).

c) Risk level

Some actions are escalated by nature , regardless of trust: transfer beyond an amount, deletion of data, engaging external communication, modification of production. We define a risk matrix :

Impact \ Reversibility Reversible Irreversible
Weak Autonomous Autonomous + enhanced logging
AVERAGE Autonomous + retrospective review Prior human approval
Pupil Prior human approval Human approval + double validation

2.3 The three climbing patterns

Pattern 1 — Pause-and-ask

The agent suspend its execution, asks a specific question to the human, waits for the answer, then continues. Synchronous from a task perspective.

Pattern 2 — Queue-for-review

The agent finished its work but the result is placed in a validation queue before publication/execution. Asynchronous, the agent is released immediately.

Pattern 3 — Fallback-to-human

The agent abandoned the task and transfers it entirely to a human, with all the accumulated context. This is the terminal safety net.

2.4 Integration of human feedback

Climbing isn’t just a safety net — it’s a learning sensor . Every human correction is data.

Four-step feedback loop:

  1. Capture : each human decision (approval, rejection, correction) is logged in a structured way: {task_id, sortie_agent, action_humaine, sortie_corrigée, motif}.
  2. Aggregation : Periodic analysis of corrections by category — what types of tasks are most often corrected? What reasons come back?
  3. Injection : the lessons return to the system through three channels, from the lightest to the heaviest:
    • System prompt : addition of rules and counterexamples taken from recurring corrections.
    • Few-shot examples / knowledge base : the pairs (case → validated correction) feed the context recovery (RAG — Retrieval-Augmented Generation , generation augmented by recovery).
    • Fine tuning (fine adjustment of the model): only if the volume of corrections is sufficient and the target behavior stable. ⚠ Expensive, to be re-evaluated each time the basic model evolves.
  4. Measure : the correction rate per category must decrease after injection. Otherwise, the loop doesn’t close — it’s feedback theater.

Educational point of emphasis: many organizations capture feedback but never inject it. The audit question to ask: “show me the last prompt modification caused by human correction, and its date”.

2.5 Management of escalation SLAs

Escalation without SLA (Service Level Agreement — service level agreement) is an escalation towards a black hole.

Components of an SLA policy:

  1. Escalation Timeout : if no human responds within the time limit, a default action applies. Three timeout strategies:

    • Fail safe (default security): the task is canceled or put on long hold — for risky actions.
    • Fail-operational : The agent applies the pre-approved conservative option — for low-stakes tasks.
    • Re-escalation : the request goes up one level (n+1 hierarchical, on-call team).

  2. Priority queues : not all climbs are equal. Typical three-level queue:

Priority Example Target deadline ⚠ (indicative)
P1 — Review Blocked transaction, customer in distress, legal risk < 15 mins
P2 — Standard Content validation, non-blocking ambiguity < 4 a.m.
P3 — Deferrable Sampling review, continuous improvement < 48 hours

  1. Routing by skill : legal escalation goes to the lawyer, not to general support. The escalation packet carries labels (domain, language, urgency) used by the router.

  2. Dashboards : escalation rate, first human response time, SLA exceedance rate, distribution by reason. These are architectural metrics, not just operational.

Block 3 — Error handling in multi-agent systems

3.1 The propagation of errors: the domino effect

In a multi-agent system, a local error becomes systemic by cascade through the orchestrator :

Sous-agent C échoue
      ↓
Agent B attend C → timeout → B échoue
      ↓
L'orchestrateur reçoit deux échecs corrélés → abandonne la mission entière
      ↓
Résultat : 45 minutes de travail des agents A, D, E jetées à la poubelle

Three anti-cascade principles:

  1. Insulation (bulkheads — watertight bulkheads) : the failure of a branch should not invalidate independent branches. The orchestrator aggregates the partial results: “4 out of 5 analyzes delivered, the tax analysis failed”.
  2. Systematic timeouts : any inter-agent call carries a maximum delay. An agent waiting indefinitely is a silent failure.
  3. Typology of errors : distinguish errors retryable (network timeout, rate limit — API rate limit, temporary overload) errors fatal (invalid entry, permission denied, business constraint violated). Retrying a fatal error wastes budget and delays escalation.
class AgentError(Exception):
    retryable: bool

class RateLimitError(AgentError):   retryable = True
class ToolTimeoutError(AgentError): retryable = True
class InvalidInputError(AgentError): retryable = False
class PolicyViolationError(AgentError): retryable = False

3.2 Retry strategies

a) The prerequisite: the idempotence of the tools

A tool is idempotent if running it twice produces the same effect as once. Without idempotence, retry is dangerous: replaying “create an invoice” after an ambiguous timeout can create a duplicate (the timeout does not say whether the first attempt was successful on the server side).

Standard technique: the key to idempotence — the client generates a unique identifier by intention of operation; the server deduplicates.

import uuid

def create_invoice(client_id: str, amount: float) -> Invoice:
    idem_key = f"inv-{uuid.uuid4()}"          # générée UNE fois par intention
    for attempt in range(3):
        try:
            return api.post("/invoices",
                            json={"client_id": client_id, "amount": amount},
                            headers={"Idempotency-Key": idem_key})
        except TransientError:
            sleep(backoff(attempt))            # backoff exponentiel + jitter
    raise ToolFailedError("create_invoice", idem_key)

def backoff(attempt: int) -> float:
    """Backoff exponentiel avec jitter (gigue aléatoire) pour éviter
    que tous les agents ne réessaient au même instant (thundering herd)."""
    import random
    return min(30, (2 ** attempt)) * random.uniform(0.5, 1.5)

Architect rule: in the specification of each tool exposed to agents, explicitly document idempotent: true/false . The orchestrator only allows automatic retry on tools marked idempotent.

b) Status checkpointing (save points)

For long tasks, saving the state at each milestone allows you to resume after failure without replaying everything :

class CheckpointedMission:
    def run(self, mission_id: str):
        state = self.store.load(mission_id) or MissionState.initial()
        for step in self.plan.steps:
            if step.id in state.completed_steps:
                continue                       # déjà fait : on saute
            result = self.execute(step, state)
            state.record(step.id, result)
            self.store.save(mission_id, state)  # checkpoint après CHAQUE jalon

The checkpoint contains: steps accomplished, artifacts produced, budget consumed, and the minimal context necessary for resumption (not the entire conversation history – summarize it).

c) Partial Rollback (partial cancellation)

When a step fails in the middle of a sequence with side effects, we cancel the effects of the sequence in progress without destroying the work of the validated sequences. Boss : compensation (each action has a registered opposite action). Inspired by the boss Saga microservices architectures:

Étape 1 : réserver stock       → compensation : libérer stock
Étape 2 : débiter compte       → compensation : recréditer compte
Étape 3 : créer expédition ✗ ÉCHEC
→ Rejouer les compensations 2 puis 1 (ordre inverse), état cohérent restauré.

3.3 The Circuit Breaker pattern (software circuit breaker)

Issue : an external service (tool API, subagent) goes down. Without protection, each task continues to call it, suffers the complete timeout, wastes tokens and latency, and overloads the already dying service.

Solution : a circuit breaker with three states, like an electrical circuit breaker:

                 N échecs consécutifs
   ┌─────────┐ ─────────────────────► ┌─────────┐
   │  FERMÉ  │                        │ OUVERT  │
   │(closed) │ ◄──── succès ────┐     │ (open)  │
   └─────────┘                  │     └────┬────┘
   appels passent               │          │ après délai de refroidissement
   normalement                  │          ▼ (cooldown)
                          ┌─────┴───────────┐
                          │   SEMI-OUVERT   │──── échec ────► retour à OUVERT
                          │   (half-open)   │
                          └─────────────────┘
                          laisse passer UN appel d'essai

import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, cooldown_s=30):
        self.failure_threshold = failure_threshold
        self.cooldown_s = cooldown_s
        self.failures = 0
        self.state = "closed"
        self.opened_at = None

    def call(self, fn, *args, **kwargs):
        if self.state == "open":
            if time.time() - self.opened_at >= self.cooldown_s:
                self.state = "half_open"       # tentative d'essai autorisée
            else:
                raise CircuitOpenError("appel rejeté : circuit ouvert")
        try:
            result = fn(*args, **kwargs)
        except TransientError:
            self._on_failure()
            raise
        self._on_success()
        return result

    def _on_failure(self):
        self.failures += 1
        if self.state == "half_open" or self.failures >= self.failure_threshold:
            self.state = "open"
            self.opened_at = time.time()

    def _on_success(self):
        self.failures = 0
        self.state = "closed"

Multi-agent specificity: place a circuit breaker by dependence (one per external tool, one per sub-agent), never a global circuit breaker. When a circuit breaker opens, the orchestrator must know this to enable graceful degradation (§3.4) rather than letting tasks crash against it.

3.3-bis Architecture model-agnostic and build by branching

In an enterprise multi-agent architecture, the model should never be the center of gravity. The center of gravity is the control layer : orchestration, permissions, tools, evaluations, logs, and common business representation. An architecture model-agnostic encapsulates each provider behind a stable contract: same message format, same output patterns, same evaluation metrics, same security rules. This makes it possible to route a task to the best model at the moment, to failover in the event of a failure, or to introduce a fine-tuned open-weights model without rewriting the agents.

Model-agnostic control layer architecture

Source: Palantir — Institutional Sovereignty in the Age of AI (2026)

In production, this encapsulation becomes a “harness” of continuous evolution: the real workloads feed in a loop the model replacement, the adjustment of the prompts and the validation of the outputs. This is what makes model liquidity operational — not an architecture slide, but a spinning loop.

Evolve agent harness: continuous loop of model swap, prompt tuning and validation

Source: Palantir — Institutional Sovereignty in the Age of AI (2026)

The same principle applies to system evolution: avoid directly modifying the production flow. The boss build by branching consists of creating a sandbox branch of an agent, prompt or tool, testing it on evals and historical traces, then promoting it only if it improves the metrics without breaking the guardrails. For agentic systems, it is the equivalent of behavioral CI/CD: sandbox → validation → promotion. The orchestrator does not trust something new because it is promising; he promotes her because she has passed the test.

Build by branching: sandbox, validation, promotion

Source: Palantir — Institutional Sovereignty in the Age of AI (2026)

Architect rule: the choice of model should be an operating variable, not a structural dependency. If changing the model requires rewriting permissions, logs or tools, the architecture is locked in the wrong place.

3.4 Graceful degradation

Principle: faced with a partial breakdown, reduce capacity rather than fail completely . The system delivers a diminished but honest service.

Degradation scale (to be designed a priori, not in panic):

Level Situation Behavior
0 — Nominal Everything works Full service
1 — Light gradient A broken enrichment tool Response without enrichment, explicit mention of the lack
2 — Strong gradient Specialized sub-agent unavailable Generic response by the main agent + warning + escalation proposal
3 — Minimal Main model unavailable Switch to smaller emergency model, restricted perimeter ⚠
4 — Honest refusal Nothing works Unavailability message + capture of the request for deferred processing

Two absolute rules:

  1. Transparency : never deliver a degraded response by presenting it as complete. “The tax analysis could not be carried out (service unavailable); here are the other 4 parts” — this is acceptable. Silence about lack is not.
  2. Degradation ≠ reduction in security : the guardrails (filters, validations, escalations) never deteriorate. We degrade the ability , not the control .

3.5 Dead Letter Queue (DLQ)

Principle: a task that has exhausted its retries and cannot be processed is not never silently deleted . It is placed in a dedicated queue – the dead letter queue – with all its diagnostic context.

Contents of a well-formed DLQ entry:

{
  "task_id": "task-8842",
  "correlation_id": "corr-a1b9f3",
  "deposited_at": "2026-07-02T14:31:07Z",
  "attempts": 3,
  "last_error": {"type": "ToolTimeoutError", "tool": "search_registry", "timeout_s": 60},
  "agent_chain": ["orchestrator", "agent-juridique", "sub-agent-registre"],
  "payload": { "...la tâche originale intacte..." },
  "state_checkpoint_ref": "ckpt://missions/8842/step-3"
}

Operation of the DLQ:

3.6 Observability: distributed tracing and correlation identifiers

In a system where a request passes through an orchestrator, three agents, and seven tool calls, answering "why is this answer wrong?" » without tracing is archeology.

a) Correlation ID (correlation ID)

A unique identifier is generated upon entry of the request and propagated through each agent, each tool call, each queue message . Every newspaper line carries it.

import contextvars, uuid

correlation_id = contextvars.ContextVar("correlation_id")

def handle_request(request):
    cid = request.headers.get("X-Correlation-ID") or f"corr-{uuid.uuid4().hex[:8]}"
    correlation_id.set(cid)
    log.info("request.received", cid=cid, task=request.task)
    return orchestrator.run(request, cid=cid)   # propagation explicite aux sous-agents

Ruler : grep corr-a1b9f3 logs/* must replenish the entirety of the process of a request. If a component “loses” the identifier, observability is broken at that specific location.

b) Distributed tracing

Beyond logs: each operation becomes a span (timed segment) attached to a trace (the complete tree of the request). De facto standard: OpenTelemetry (open observability specification).

Trace corr-a1b9f3 (durée totale : 42,3 s)
└── span orchestrator.run                    [42,3 s]
    ├── span planner.plan                    [3,1 s]  ✓
    ├── span agent-juridique.analyze         [18,2 s] ✓
    │   ├── span tool.search_registry        [12,0 s] ✓  ← goulot identifié
    │   └── span llm.generate                [5,9 s]  ✓
    ├── span agent-fiscal.analyze            [17,8 s] ✗ TIMEOUT
    │   └── span tool.tax_api                [15,0 s] ✗  ← cause racine
    └── span redactor.merge                  [2,4 s]  ✓ (mode dégradé niveau 1)

LLM agent specific span attributes to capture: model used, input/output tokens, estimated cost, tool name, escalation decision, circuit breaker status, system prompt version. These attributes transform tracing into a tool for cost and quality optimization, not just debugging.

Architecture audit question (to be asked as is during certification interview): “Show me the full trail of a failed request last week, from point of entry to DLQ. » If the team can't, observability is declarative, not real.

Summary: the architect’s grid

Domain Design question Deliverable artifact
Decomposition Vertical, horizontal, recursive — and why? Architecture diagram + justification of the 3 criteria
Escalation What triggers, what patterns, what SLAs? Escalation policy + risk matrix
Errors What happens when X falls? (for each X) Failure mode map + degradation scale
Retry Which tools are idempotent? Annotated tool specification
Observability Can we replay the story of a request? Correlation ID propagation scheme

Final message to future certified: a multi-agent system is not judged by its behavior when everything is going well – any demo gets there. He is judged on his behavior when a tool timeouts, when an agent hallucinates, when a human does not respond. The certification evaluates precisely this: the architecture of bad days .

Instructor Notes