# Applied AI — Advanced Level — Session 4
# Exercises: Multi-agent architecture

**Instructor:** Yann Isola
**Total duration:** 90 minutes
**Modality:** individual or pair — the answers are intended for the instructor

---

## Exercise 1 — Choice of decomposition strategy (25 min)

### Context

You are a solutions architect at a broker in tokenized financial instruments. Three projects are arriving simultaneously. For **each**, you must choose a decomposition strategy (vertical, horizontal, recursive, hybrid — or *no decomposition*) and justify it with the three decision criteria (complexity threshold, token budget, specialization benefit).

### Case A — Product sheet generator

A repetitive task: from a 2-page technical sheet, generate a 300-word commercial description in a defined tone. Volume: 500 files/day. The technical sheet + the prompt are largely in context.

### Case B — Due diligence of a counterparty

For each new counterparty: legal analysis (social structure, licenses), financial analysis (balance sheets, ratios), reputational analysis (press, sanctions), technical analysis (asset conservation infrastructure). Then production of a summary memo with GO/NO-GO recommendation. Each component requires voluminous documents and a distinct specialized vocabulary.

### Case C — Documentary migration of a fund

A collection transmits a tree structure of ~4,000 documents (unpredictable depth and structure: nested files, heterogeneous formats, mixed languages). Each document must be classified, summarized, indexed and linked to a regulatory framework. The structure of the tree is only known at runtime.

### Expected deliverables (per case)

1. Chosen strategy (one sentence).
2. Justification by the 3 criteria (table).
3. Architectural diagram (ASCII or drawing): agents, flows, points of convergence.
4. For case C only: the **three mandatory safeguards** of your architecture and their proposed values.

---

### Instructor answer key — Exercise 1

**Case A — No decomposition.**
- Complexity: a single homogeneous step, no branching — far below the threshold.
- Token budget: the form + the prompt fit comfortably in the context.
- Specialization: a single field; a single, well-designed prompt is enough.
- **Case trap:** the volume (500/day) pushes certain participants towards multi-agents. The volume is a problem of *instance parallelization* (500 executions of the same agent), not of *task decomposition*. Point to be corrected explicitly if the error appears.
- Acceptable bonus: a lightweight downstream verifier (2-stage vertical pipeline) if the participant justifies it by a quality issue — but he must recognize the additional cost of 2× on 500 executions/day.**Case B — Hybrid: horizontal for the analysis, vertical for the whole.**
- Expected architecture: orchestrator → 4 domain agents **in parallel** (legal, financial, reputational, technical) → fan-in towards a writer agent → verification agent (or human review, GO/NO-GO being a high risk decision → escalation by nature, link with chapter 9).
- Complexity: > 5 stages of different natures and domains → above the threshold.
- Tokens: each component uses voluminous documents → a single agent would saturate its window.
- Specialization: maximum — distinct vocabularies and tools per domain.
- **Excellent reflex to promote:** the participant who provides for the reconciliation of inter-domain contradictions (the legal so-called GO, the reputational so-called NO-GO) and the isolation of breakdowns (deliver 3 out of 4 parts with explicit mention).

**Case C — Recursive.**
- Unpredictable and fractal structure (nested files of unknown depth): this is the textbook case of recursion — each file agent decomposes according to what it discovers.
- The three mandatory safeguards:
1. `max_depth`: 2–3 levels (defensible value if justified);
2. inherited token budget: fraction of the parent budget per child (e.g.: 30%);
3. structured result contract (JSON: status, artifacts, cost) for each sub-agent.
- Acceptable bonus safeguard: overall ceiling of simultaneous sub-agents (rate limit protection), checkpointing per file processed (link chapter 10).

**Indicative scale (out of 20):** Case A: 5 pts (including 2 for resisting over-engineering) · Case B: 8 pts (including 3 for the correctly articulated hybrid) · Case C: 7 pts (including 3 for the encrypted safeguards).

---

## Exercise 2 — Designing an escalation policy (30 min)

### Context

A multi-agent customer service assistant operates for a digital asset custody platform. It may: answer questions (products, fees, procedures), modify non-sensitive profile information, initiate account recovery procedures, and prepare (but not execute) withdrawal instructions.

Three recent incidents motivate an overhaul of the escalation policy:

- **Incident 1:** the agent guided a customer through an account recovery procedure while social engineering signals were present (artificial urgency, inconsistencies in the information provided). Nobody was alerted.
- **Incident 2:** a pointed tax question (multiple tax residence) received a confidently invented answer. The customer made a decision based on this.
- **Incident 3:** requests escalated to the compliance team accumulate without processing — some waited 6 days; two customers left.

### Work requested

1. **Trigger matrix (8 pts)** — For each of the agent's 4 capabilities, define the escalation triggers according to the three families (trust below threshold / outside perimeter / risk level). Present in table form. Each incident must be covered by at least one trigger.
2. **Choice of patterns (6 pts)** — For each trigger, associate the appropriate pattern (pause-and-ask / queue-for-review / fallback-to-human) and justify in one sentence.
3. **SLA policy (6 pts)** — Design the response to incident 3: priority levels, target deadlines, timeout action by level (fail-safe / fail-operational / re-escalation), and **two management metrics**.### Constraint

Your policy must remain actionable: if everything escalates in P1 to a human, you have failed. Explicitly state what remains **standalone**.

---

### Instructor answer key — Exercise 2

**1. Trigger matrix – expected elements:**

| Capacity | Trust | Outside the scope | Risk |
|---|---|---|---|
| Product/cost questions | Separate judge, calibrated threshold (ex. 0.75) → escalation if below | Tax/legal/investment advice detected → escalation (covers **incident 2**) | Low → standalone by default |
| Non-sensitive profile modification | — | Request sliding to sensitive data (IBAN, 2FA) → escalation | Low/reversible → standalone + logging |
| Account Recovery | Inconsistencies in verifications → escalation | — | **High by nature**: fraud signal detector (urgency, pressure, inconsistencies) → mandatory escalation (covers **incident 1**) |
| Preparing for removal | — | — | High/irreversible downstream → systematic prior human approval above an amount threshold |

Key points to check: (a) incident 2 is handled by *out of scope* (regulated tax advice), not just trust — an assertively invented response precisely has high self-reported trust, hence the need for the scope classifier and/or a separate judge; (b) Incident 1 requires a trust-independent *risk* trigger.

**2. Expected patterns:**
- Out-of-scope tax question → **fallback-to-human** with full context package (the client expects a real answer, not a deferred validation).
- Fraud signals on account recovery → **pause-and-ask** to the fraud team (the agent suspends the procedure - do not complete it then have it reread); also admissible: immediate fallback. To sanction: queue-for-review (the procedure would already be completed at the time of the review).
- Preparation of withdrawal beyond the threshold → **queue-for-review** (the agent prepares, the human approves before execution) — this is the canonical use case of this pattern.
- Low confidence on product question → queue-for-review (low-stakes response) or pause-and-ask depending on justification.

**3. SLA — typical response:**
- P1 (< 15–30 min): suspicion of fraud, compromised account. Timeout → **re-escalation** to on-call + **fail-safe**: freezing of the procedure while waiting.
- P2 (< 4 working hours): regulated questions, withdrawals awaiting approval. Timeout → re-escalation n+1; withdrawal remains blocked (fail-safe).
- P3 (< 48 h): sampling reviews, content corrections. Timeout → fail-operational (pre-approved conservative option applies).
- Metrics: human first response time by priority, SLA exceeded rate, escalation rate by reason, approval rate without modification (over-escalation signal). Two is enough, four = excellent.
- **Autonomy preserved:** high-confidence product questions and non-sensitive modifications remain autonomous — essential to validate the usability constraint.

---

## Exercise 3 — Implementing error handling (35 min)

### Context

You receive the skeleton of an orchestrator that calls three subagents in parallel and then merges their results. In production, the `enrichisseur` agent API is experiencing intermittent outages that are currently causing entire missions to fail.```python
# --- CODE EXISTANT (défaillant) ---
def run_mission(task):
    r1 = agent_analyste.run(task)        # fiable
    r2 = agent_enrichisseur.run(task)    # pannes intermittentes !
    r3 = agent_redacteur_notes.run(task) # fiable
    return merge(r1, r2, r3)             # KeyError si r2 manque
```### Work requested

Rewrite `run_mission` and its infrastructure to integrate, in this order:

1. **(6 pts) Typology + retry with backoff** — Classify errors (retryable / fatal). Implement a retry (max 3 attempts) with exponential backoff and jitter, applied **only** to retryable errors. Is the call to `agent_enrichisseur` replayable without caution? Justify in one sentence in a comment (hint: idempotence).
2. **(6 pts) Circuit breaker** — Implement a circuit breaker (threshold: 5 failures, cooling: 30 s) around `agent_enrichisseur` only. All three states must be explicit in the code.
3. **(4 pts) Graceful degradation** — If the enricher is unavailable (open circuit or exhausted retries), the mission must still **succeed** with a partial result, carrying an explicit mention of the missing component. `merge` should never raise `KeyError` again.
4. **(4 pts) DLQ + correlation** — Any definitively failed subtask goes into a dead letter queue with: correlation ID, number of attempts, last error, checkpoint reference. The correlation ID crosses all mission logs.

### Constraints

- Standard Python only (no external library); `time`, `random`, `uuid`, `json` allowed.
- The code must be executable as is with simulated agents (provide a stub `FlakyAgent` which randomly fails ~40%).
- Test: launch 20 missions and display the final table (complete successes / degraded successes / DLQ entries / circuit openings).

---

### Instructor answer key — Exercise 3 (reference solution)```python
import time, random, uuid, json

# ---------- 1. Typologie d'erreurs ----------
class AgentError(Exception): retryable = False
class TransientError(AgentError): retryable = True    # timeout, rate limit
class FatalError(AgentError): retryable = False       # entrée invalide, policy
class CircuitOpenError(AgentError): retryable = False

def backoff(attempt):
    return min(8, 2 ** attempt) * random.uniform(0.5, 1.5)

def with_retry(fn, cid, name, max_attempts=3):
    # L'agent enrichisseur est un appel de LECTURE (pas d'effet de bord) :
    # il est donc idempotent par nature → retry sûr. S'il créait des
    # ressources, il faudrait une clé d'idempotence avant tout retry.
    last = None
    for attempt in range(max_attempts):
        try:
            return fn()
        except TransientError as e:
            last = e
            log(cid, f"{name} tentative {attempt+1} échouée : {e}")
            time.sleep(backoff(attempt) * 0.01)  # ×0.01 pour le test en classe
    raise last

# ---------- 2. Circuit breaker ----------
class CircuitBreaker:
    def __init__(self, threshold=5, cooldown=30 * 0.01):  # cooldown réduit pour la démo
        self.threshold, self.cooldown = threshold, cooldown
        self.failures, self.state, self.opened_at = 0, "closed", None
        self.opens = 0

    def call(self, fn):
        if self.state == "open":
            if time.time() - self.opened_at >= self.cooldown:
                self.state = "half_open"
            else:
                raise CircuitOpenError("circuit ouvert")
        try:
            result = fn()
        except TransientError:
            self.failures += 1
            if self.state == "half_open" or self.failures >= self.threshold:
                self.state, self.opened_at = "open", time.time()
                self.opens += 1
            raise
        self.failures, self.state = 0, "closed"
        return result

# ---------- Stubs ----------
class FlakyAgent:
    def __init__(self, name, fail_rate=0.0):
        self.name, self.fail_rate = name, fail_rate
    def run(self, task):
        if random.random() < self.fail_rate:
            raise TransientError(f"{self.name} : timeout simulé")
        return {"agent": self.name, "content": f"résultat({task})"}

def log(cid, msg): print(f"[{cid}] {msg}")

# ---------- 3 & 4. Orchestrateur avec dégradation + DLQ ----------
DLQ = []
breaker = CircuitBreaker()

def run_mission(task):
    cid = f"corr-{uuid.uuid4().hex[:8]}"
    results, missing = {}, []

    results["analyste"] = agent_analyste.run(task)          # fiable (simplifié)
    results["notes"] = agent_redacteur_notes.run(task)

    try:
        results["enrichisseur"] = with_retry(
            lambda: breaker.call(lambda: agent_enrichisseur.run(task)),
            cid, "enrichisseur")
    except AgentError as e:
        missing.append("enrichisseur")
        DLQ.append({
            "task_id": task, "correlation_id": cid,
            "attempts": 3 if not isinstance(e, CircuitOpenError) else 0,
            "last_error": type(e).__name__,
            "checkpoint_ref": f"ckpt://{task}/partial",
        })
        log(cid, f"enrichisseur abandonné → DLQ ({type(e).__name__})")

    return merge(results, missing, cid)

def merge(results, missing, cid):
    out = {"correlation_id": cid, "sections": results, "complete": not missing}
    if missing:
        out["avertissement"] = (f"Volets indisponibles : {', '.join(missing)} "
                                "(service en panne) — résultat partiel.")
    return out

# ---------- Test : 20 missions ----------
agent_analyste = FlakyAgent("analyste")
agent_enrichisseur = FlakyAgent("enrichisseur", fail_rate=0.4)
agent_redacteur_notes = FlakyAgent("notes")

full = degraded = 0
for i in range(20):
    m = run_mission(f"task-{i}")
    full += m["complete"]; degraded += not m["complete"]

print(f"\nComplets: {full} | Dégradés: {degraded} | DLQ: {len(DLQ)} | "
      f"Ouvertures circuit: {breaker.opens}")
```**Scoring points to check:**
- The retry only catches `TransientError` (catching `Exception` = -2 pts: we would replay fatal errors).
- The idempotence comment is present and correct (reading without side effects → replayable; otherwise idempotence key).
- The circuit breaker *only* surrounds the enricher; the three states appear; the half-open state falls back to open on failure.
- The nesting order retry(breaker(call)) is the right debate: breaker *inside* retry means that attempts rejected by the open circuit count as immediate failures — acceptable if the student justifies it; the opposite (retry inside the breaker) counts 3 attempts as 1 circuit failure — also acceptable. **The important thing is that the choice is conscious and commented on.**
- `merge` produces an honest result: flag `complete`, explicit warning — never a silently absent section.
- The DLQ entry contains the 4 required fields; `cid` appears in each log line.

**Extension for the quick:** add a real checkpoint (dict serialized in JSON) and a function `replay_dlq()` which replays the inputs after "repair" (fail_rate set to 0) — demonstrates the complete DLQ cycle → correction → replay.

---

*End of exercises — Session 4, advanced level. Answer keys should not be distributed before the feedback session.*