title: "Applied AI — Advanced S4: Multi-Agent Architecture"
author: "Yann Isola"
theme: applied-ai
palette: ["#1A2230", "#0F7A6C", "#B4612A", "#E9F6F3", "#F4F7F6"]

Slide 1 — Title

Multi-agent architecture

Applied AI — Advanced level · Session 4

Yann Isola — Preparation Claude Certified Architect

Decomposition · Escalation & HITL · Error handling

Slide 2 — Session Objectives

At the end of this session, you will know:

1. Choose a decomposition strategy (vertical / horizontal / recursive) and justify it
2. Design a complete Human-in-the-Loop escalation policy (triggers, patterns, SLAs)
3. Implement error patterns: idempotent retry, circuit breaker, graceful degradation, DLQ
4. Instrument a multi-agent system (distributed tracing, correlation IDs)

Common thread: the architecture of bad days — a system is judged when everything breaks, not when everything is going well.

Slide 3 — Why decompose? The three walls

A single agent facing a complex task:

• Context wall — the context window is over; saturating it degrades the quality
• Wall of consistency — the longer the task, the more the drift increases
• Wall of specialization — a general prompt < specialized prompts

But: each agent added = latency + tokens + operational complexity.

The question is never “can we decompose?” but “does the gain exceed the cost?”

Slide 4 — Vertical decomposition (by depth)

Sequential steps of different types:

Planning       →  Execution  →  Verification
 (the plan =       (tools,        (audit against the
  auditable         artifacts)     success criteria)
  contract)

Short and targeted prompts · independent verifier (anti-bias self-assessment)
Cumulative latency · plan that may become obsolete → replanning loop

Slide 5 — Vertical: the code

plan   = planner.plan(task)              # plan + success criteria
result = executor.execute(plan)
for attempt in range(max_repairs):       # bounded repair loop
    report = verifier.check(result, plan.success_criteria)
    if report.ok:
        return Result("success", result)
    result = executor.repair(result, report.issues)
return Result("needs_human_review", result)   # → escalation (chap. 9)

Note: the needs_human_review output — decomposition natively interfaces with escalation.

Slide 6 — Horizontal breakdown (by domain)

Each agent has a domain, processed end-to-end:

              Orchestrator
      ┌──────────┼──────────┬──────────┐
      ▼          ▼          ▼          ▼
    Legal       Tax       Risk      (parallel !)
      └──────────┴──────────┴──────► Synthesis (fan-in)

Latency ≈ slowest agent (not the sum) · max specialization · fault isolation
Inter-domain contradictions → obligatory reconciliation step

Slide 7 — Recursive decomposition (tree)

Each agent can decompose its own subtask:

Root   ── Agent A ── Sub-agent A1
      │           └─ Sub-agent A2 ── A2a
      └── Agent B ── Sub-agent B1

Ideal for: unpredictable or fractal structures (monorepos, documentary trees)

MANDATORY guardrails:

1. max_depth (2–3 levels )
2. Inherited token budget per level (e.g.: 30% of parent)
3. Structured results contract: {status, artifacts, cost} — never free text

Slide 8 — When to decompose? The 3 criteria

Criterion	Question	Indicative threshold
Complexity	> 5–7 heterogeneous stages?	Yes → decompose
Budget tokens	Context > ~60–70% of window?	Yes → decompose
Specialization	Measurable quality gain per domain?	Gain > additional cost → decompose

Golden rule: the most robust architecture is the one that does not exist.
If one agent is enough, one agent is enough.

Slide 9 — Certification pitfall #1

“The task is executed 1,000 times per day, should we break it down? "

NO.

• High volume = instance parallelization problem (N copies of the same agent)
• Decomposition = task structure problem

The two combine, but never merge.
Vertical/horizontal discriminant: sequence of different natures vs parallelism of domains.

Slide 10 — Climbing: an architectural requirement

No agent system achieves 100% reliable autonomy.

HITL (Human-in-the-Loop): mechanisms by which a human validates, corrects or resumes the work of an agent.

A system without explicit escalation still escalates —
but in chaos mode: tickets, incidents, loss of confidence.
Design exit paths from day 1.

Slide 11 — Trigger 1: confidence below the threshold

CONFIDENCE_THRESHOLD = 0.75   #  calibrate on real data

if judge_agent.evaluate(answer).confidence < CONFIDENCE_THRESHOLD:
    escalate(task, reason="low_confidence")

3 best practices:

• Agent separate judge (self-assessment is complacent)
• Threshold calibrated on labeled validation set (accuracy curve / escalation rate)
• Monitor the drift of the escalation rate in production

Slide 12 — Trigger 2: outside the perimeter

The agent recognizes that the request is outside its authorized domain:
subject not covered · regulated advice (legal, tax, medical) · circumvention

Defense in depth (2 layers):

1. Perimeter classifier upstream (small, quick model)
2. Self-detection instruction in the system prompt

Trap #2: a guaranteed hallucination has high self-declared confidence.
The confidence threshold alone does not catch it → hence this independent trigger.

Slide 13 — Trigger 3: risk level

Some actions escalate by nature, regardless of trust.

Risk matrix:

Examples: transfers above an amount, deletion of data, modification of production.

Slide 14 — Pattern 1: Pause-and-ask

The agent suspends its execution → specific question → waits → resume.

• Usage: blocking ambiguity, one-off decision during the task
• Requirement: agent state serializable (human can respond in 4 hours)
• Risk: accumulation of suspended tasks → mandatory timeout

Timeline: work → pause → human replies → resume → end

Slide 15 — Pattern 2: Queue-for-review

The agent finishes its work → the result waits in a validation queue before publishing/executing. The agent is released immediately.

• Use: high-stakes content, preparation of actions (e.g. withdrawal instructions)
• Human processes in batches with a review interface (diff, approve/reject)
• Key metric: approval rate without modification > ~98% sustainably → consider autonomy (with sampling)

Timeline: complete work → queue → human validates → publication

Slide 16 — Pattern 3: Fallback-to-human

The agent gives up and transfers entirely to a human. The terminal net.

Golden rule: humans NEVER start from zero.

The transfer package contains:

• the initial request
• the history of the agent's actions
• the hypotheses tested
• the precise reason for the climbing

A transfer without context destroys the value of agent work.

Slide 17 — Pause-and-ask vs Queue-for-review

The most common mistake on the exam:

	Pause-and-ask	Queue-for-review
Is the agent finished?	No — suspended during	Yes — validation after
Synchronous?	Yes (for the task)	No (asynchronous)
Typical case	Blocking ambiguity	Approval before execution

Discriminating example: fraud signals during account recovery → pause-and-ask (especially not finishing the procedure and then having it reread!).

Slide 18 — The human feedback loop

Each human correction = learning data.

1. CAPTURE     {task_id, agent_output, human_action, correction, reason}
2. AGGREGATION which categories are corrected most? which reasons?
3. INJECTION   system prompt → examples/RAG → fine-tuning ( costly, last resort)
4. MEASURE     is the correction rate dropping? otherwise = feedback theatre

Audit Question: “Show me the last prompt change caused by human correction — and the date.”

Slide 19 — Escalation SLA

Escalation without SLA (Service Level Agreement) = escalation towards a black hole.

Priority	Example	Target deadline	At timeout
P1	Fraud, customer in distress, legal risk	< 15 mins	Re-escalation + fail-safe (freeze)
P2	Validations, regulated questions	< 4 hrs	Re-escalation n+1
P3	Sampling Reviews	< 48 hours	Fail-operational (conservative default)

• Routing by skill (legal matters go to the lawyer) + dashboards (escalation rate, first response time, overruns).

Slide 20 — Mistakes: the domino effect

Sub-agent C fails
  → Agent B waits for C → timeout → B fails
    → The orchestrator receives 2 failures → abandons EVERYTHING
      → 45 min of work by A, D, E: lost

3 anti-cascade principles:

1. Isolation (bulkheads): a falling branch does not invalidate the others
2. Systematic timeouts on any inter-agent call
3. Typology: retryable errors (timeout, rate limit) ≠ fatal (invalid entry, policy)

Slide 21 — Retry: idempotence first

A timeout is ambiguous: the 1st attempt may have succeeded on the server side.
Replaying “create an invoice” without caution = duplicate.

idem_key = f"inv-{uuid.uuid4()}"        # ONE key per INTENT
for attempt in range(3):
    try:
        return api.post("/invoices", json=payload,
                        headers={"Idempotency-Key": idem_key})
    except TransientError:
        sleep(backoff(attempt))          # exponential + jitter

Architect rule: each tool documents idempotent: true/false. Auto retry only on idempotent tools. Backoff with jitter (anti thundering herd).

Slide 22 — Checkpointing & partial rollback

Checkpointing — save the state at each milestone → restart without replaying everything:

if step.id in state.completed_steps: continue   # already done
result = execute(step); state.record(step.id, result)
store.save(mission_id, state)                    # after EACH milestone

Partial rollback (Saga boss) — each action has its compensation:

reserve stock → debit account → create shipment ✗
   compensations in REVERSE order: re-credit, then release the stock

Slide 23 — Circuit breaker: the 3 states

              N consecutive failures
  ┌────────┐ ───────────────────► ┌────────┐
  │ CLOSED │                      │  OPEN  │  immediate reject
  │        │ ◄─ success ──┐       │        │  (fail fast)
  └────────┘              │       └───┬────┘
  normal calls            │           │ after cooldown
                    ┌─────┴───────────▼──┐
                    │     HALF-OPEN      │─ failure ─► OPEN
                    │  (1 trial call)    │
                    └────────────────────┘

Threshold: 5 failures · Cooldown: 30 s (to be calibrated)
Multi-agents: one circuit breaker PER dependency — never global. Opening → orchestrator activates graceful degradation.

Slide 24 — Graceful degradation

Reduce capacity rather than fail completely.

Level	Location	Behavior
0	Nominal	Full service
1	KO Enrichment Tool	Answer without, missing mentioned
2	Specialized sub-agent KO	Generic response + warning
3	Main model KO	Emergency model, restricted perimeter
4	All KO	Honest refusal + capture for deferred

2 absolute rules: transparency (never present a gradient as complete) · we degrade the capacity, never the control (untouchable safeguards).

Slide 25 — Dead Letter Queue (DLQ)

A task that has exhausted its retries is never silently deleted → dead letter queue, with all the diagnostic context:

{ "task_id": "task-8842", "correlation_id": "corr-a1b9f3",
  "attempts": 3, "last_error": {"type": "ToolTimeoutError"},
  "agent_chain": ["orchestrator", "agent-legal", "sub-registry"],
  "state_checkpoint_ref": "ckpt://missions/8842/step-3" }

Operation: deposit alert (DLQ which grows silently = anti-boss) · replay after correction (thanks to idempotency keys + checkpoints) · trend analysis (80% of the same timeout = infrastructure problem, no agents).

Slide 26 — Observability: correlation ID

A unique identifier generated on entry, propagated everywhere: agents, tools, files, logs.

cid = headers.get("X-Correlation-ID") or f"corr-{uuid4().hex[:8]}"
log.info("request.received", cid=cid)
orchestrator.run(request, cid=cid)   # explicit propagation

Liquid test: grep corr-a1b9f3 logs/* must reconstruct the entire course.
A component that “loses” the ID = broken observability at that specific location.

Slide 27 — Distributed tracing (OpenTelemetry)

Each operation = a span (timed segment) in a trace (demand tree):

Trace corr-a1b9f3                          [42,3 s]
├── planner.plan                           [3,1 s]  ✓
├── agent-juridique.analyze                [18,2 s] ✓
│   └── tool.search_registry               [12,0 s] ← goulot !
├── agent-fiscal.analyze                   [17,8 s] ✗ TIMEOUT ← cause racine
└── redactor.merge                         [2.4 s]  ✓ (degraded lvl 1)

LLM specific attributes: model, in/out tokens, cost, escalations, circuit breaker status, prompt version. → cost/quality optimization tool, not just debugging.

Slide 28 — The architect’s grid

Slide 29 — Exam pitfalls to remember

1. Volume ≠ decomposition → parallel instances, no multi-agents
2. Confidence ≠ perimeter → the assured hallucination passes the confidence filter
3. Pause-and-ask ≠ queue-for-review → suspended during vs committed after
4. Retry without idempotence = duplicates → idempotence key first
5. Half-open: failure in semi-open → direct return to OPEN
6. Degrade capacity, never control → untouchable guardrails
7. Cross-questions: “5 failures of a sub-agent?” → circuit breaker THEN climbing

Slide 30 — Next steps

Today:

• 3 exercises: choice of decomposition · escalation policy · implementation errors
• Interactive demo: decomposition visualizer, climbing builder, circuit breaker simulator
• Validation quiz (10 questions — objective 8/10)

Session 5: evaluation and benchmarking of agentic systems

To remember in one sentence: a multi-agent system is judged on
the architecture of its bad days.

Applied AI — Yann Isola