Français
Applied AI · Advanced 🔴 · Session 9
📝 Teacher's Guide
← Return to program 📄 Source .md

Trainer's Guide — Advanced Level, Session 9

“Certification scenarios”

Program : Applied AI — Yann Isola Audience : Solutions architects preparing for certification Claude Certified Architect Duration : 2:00 a.m. (+ 10 min recommended break halfway through) Prerequisites: Sessions 1 to 8 of the advanced level (API Claude, tool use, MCP, Claude Code, advanced prompt engineering, context & reliability). This session does not present any new concept : it assembles everything that has been seen, from the exact angle of the examination. Material : interactive web page of the session (webpage/index.html — scenario analyzer, domain coverage tracker, timed exam simulator), projector, score sheet per participant.

Educational objectives

At the end of the session, each participant knows:

  1. Describe the format of the exam: 60 questions, 90 minutes ⚠, MCQ (MCQ = Multiple Choice Questionnaire) and questions based on scenarios, success threshold 72% ⚠ (i.e. 720 on a scale of 100 to 1000), 4 scenarios drawn from 8, no penalty for a wrong answer.
  2. Map the 5 areas of the exam and their weighting: Architecture & agent orchestration (27%), tool design & MCP integration (18%), Configuration & workflows Claude Code (20%), Prompt engineering & structured output (20%), Context management & reliability (15%).
  3. Analyze each of the 8 scenarios of the exam: identify the expected architectural decisions, the recurring pitfalls and the areas called for.
  4. Apply the “root cause → least effort → guarantee required” reading grid to eliminate distractors from a certification multiple choice question.
  5. Self-assess its coverage of the 5 areas and build a revision plan prioritized by weighting.
  6. Manage exam time: 90 seconds per question on average ⚠, marking and second pass strategy.

⚠ Session convention: all figures marked ⚠ (exam format, number of questions, duration, threshold, weightings) are volatile . They reflect the exam guide at the time of writing. Absolute reflex: check the official Anthropic certification page before booking the exam.

Timed plan

Block Duration Content
0. Opening 5 mins Exam format, rules of the game, session method
1. The MCQ reading grid 10 mins Anatomy of a certification question, method of elimination
2. Scenarios 1 & 8 — Customer support & agentic tools 20 mins Routing, escalation, preconditions, tool selection and chaining
3. Scenarios 3 & 7 — Multi-agents & Conversational AI 20 mins Coordinator/subagents, consensus, stateful sessions, memory
Break 10 mins
4. Scenarios 2, 4 & 5 — Claude Code (dev, productivity, CI/CD) 25 mins CLAUDE.md, slash commands, plan mode, headless, PR review
5. Scenario 6 — Extraction of structured data 15 mins JSON schemas, validation, edge cases, trust calibration
6. Exam simulator 20 mins Timed mini-exam (web page), collective correction
7. Revision plan & fence 5 mins Domain tracker, prioritization, review logistics

Block 0 — Opening (5 min)

Tagline message: “You have eight sessions of material in your head. The exam won't ask you to recite it: it will put you in the shoes of an architect facing a dysfunctional system, with four plausible fixes — three of which are well-constructed traps. Today, we learn to think like the author of the question. »

The format, black on white

Project and comment on this table (volatile figures ⚠):

Setting Value ⚠
Number of questions 60
Duration 90 minutes (or 90 seconds/question on average)
Kind MCQ, 1 correct answer out of 4 + scenario-based questions
Rating Scale 100–1000, threshold 720 (≈ 72 %)
Penalty for error None — answer all questions, always
Scenarios 4 drawn randomly from 8

Three practical consequences to hammer home:

  1. No penalty → we never leave a question empty. Even at random, there is a 25% chance of winning.
  2. 4 scenarios out of 8 → we cannot make a dead end. Each scenario must be mastered, because we don't know which ones will fall.
  3. 72% threshold → we are entitled to ~16 errors out of 60. This is comfortable if the 3 heaviest areas (27 + 20 + 20 = 67% of the exam) are solid.

The 5 areas and their weight

Domain Weighting ≈ Questions out of 60 ⚠
1. Architecture & agent orchestration 27 % ~16
2. Tool design & MCP integration 18 % ~11
3. Setup & workflows Claude Code 20 % ~12
4. Prompt engineering & structured output 20 % ~12
5. Context management & reliability 15 % ~9

Strategy point: Domain 1 weighs almost twice as much as Domain 5. An hour of review on agent orchestration “returns” statistically more than an hour on context management — but be careful, the domains intersect in the scenarios: a “customer support” question can evaluate Domain 5 (escalation).

Block 1 — The MCQ reading grid (10 min)

1.1 Anatomy of a certification question

Each question follows the same skeleton:

  1. Situation — a system in production with a measured symptom (“in 12% of cases, the agent jumps get_customer … ») ;
  2. Question — almost always some variation of “what change is the most effective ? » or “what is the first stage ? » ;
  3. Four options — one correct, three distractors constructed according to identifiable recipes.

Emphasize the trap words in the statement: “most effective”, “the first step”, “with the least effort”, “the best”. These are not ornaments: they mean that several options may be technically valid , and which is decided by the cost/impact ratio or by the logical order of intervention.

1.2 The four families of distractors

Have the room build this table (she saw enough questions in previous sessions to fill it):

Distractor family Signature Typical example
The magic prompt Solve by prompt which requires a deterministic guarantee “Improve the prompt system” in the face of a critical business rule
Over-engineering A heavy component (ML classifier, routing layer) for a simple problem “Train a separate classifier” when explicit criteria are sufficient
The invented functionality A flag, variable or file that does not exist CLAUDE_HEADLESS=true , --batch , .claude/config.json
The symptom, not the cause A plausible fix that addresses another problem “Sentiment Analysis” for an escalation calibration problem

1.3 The reading grid in three questions

To be applied systematically, in order:

  1. What is the root cause? (not the symptom - reread the situation, the numerical data points to the cause)
  2. Does the rule require a deterministic guarantee or is probabilistic compliance sufficient? (deterministic → programmatic hooks/preconditions; probabilistic → prompt/few-shot)
  3. Of the options that address the cause, which has the best effort/impact ratio? (certification rewards the simplest correction that resolves the problem)

Chalkboard demo with Question 1 of the exam guide (preconditions blocking process_refund ): unfold the grid step by step. Response A (programmatic precondition) comes out in 30 seconds, because the situation says “incorrect reimbursements” = critical business rule = deterministic guarantee required = prompt options (B, C) fall automatically.

Block 2 — Scenarios 1 & 8: Customer support & agentic tools (20 min)

2.1 Scenario 1 — Customer Support Agent

The guide’s statement: agent built with the Claude Agent SDK (SDK = Software Development Kit) to manage returns, billing disputes and account issues. MCP tools (MCP = Model Context Protocol): get_customer , lookup_order , process_refund , escalate_to_human . Objective: resolution at first contact > 80% with appropriate escalation.

Domains summoned: 1 (orchestration, preconditions), 2 (tool descriptions, structured errors), 5 (escalation, ambiguity).

The expected architectural decisions — to be displayed on the board:

Problem in the scenario Architect response expected Why not the alternatives
Agent skips ID verification before refund Programmatic precondition (hook) which blocks process_refund as long as get_customer did not return a verified ID The prompt = probabilistic conformity; money demands determinism
Bad routing between get_customer And lookup_order Enrich tool descriptions : input formats, examples, limitations, when to use one vs the other Descriptions are the #1 selection mechanism; the few-shot comes after
Poorly calibrated escalation (escalates simple cases, keeps complex cases) Explicit escalation criteria + few-shot examples in the system prompt Self-perceived confidence is unreliable; the feeling ≠ complexity
Multiple customer matches for one name Request additional credentials to the user No heuristic assumptions about identity
Explicit request to speak to a human Immediate escalation , without prior investigation Delaying an explicit request degrades satisfaction and violates the escalation contract

Point of attention feeling: the scenario mentions sentiment analysis. On examination, it almost always appears distractor : the feeling (anger, frustration) is a tone signal, not a reliable indicator of the complexity of the case nor an escalation criterion in itself. An irate customer may have a trivial problem; a calm customer, an inextricable political exception. Good practice: feeling can modulate your of the response, but the escalation is decided on explicit policy criteria .

Transfer protocol (handoff): when escalating, produce a structured summary — client ID, reason, actions already attempted, recommended action. A classic exam question opposes “transferring the raw conversation” (bad: the human must reread everything) to “structured summary” (good).

2.2 Scenario 8 — Agentic AI tools

Honest context to give: this scenario is reported by candidates but less documented in community guides ⚠. It covers three transversal skills: tool selection, chaining, error recovery . Good news: everything is already covered by Domains 1 and 2 — it’s a recombination.

Tool selection — the rules to know by heart:

Chaining — two patterns:

Error recovery — the taxonomy to recite:

Error category Example Retryable? Good response from the tool
Transient timeout, service unavailable Yes errorCategory: "transient", isRetryable: true
Validation invalid input format Yes, after correcting the entry Message specifying the expected format
Job reimbursement above policy threshold No retryable: false + readable explanation
Permission Access denied No (climb) Distinguish from a valid empty result

The gold trap of Domain 2: distinguish “access failure” (you have to decide whether to retry or escalate) from “valid empty result” (the search worked, there are just no matches). A tool that returns an empty set in case of timeout masks failure as success — definite anti-model under examination.

Block 3 — Scenarios 3 & 7: Multi-agents & Conversational AI (20 min)

3.1 Scenario 3 — Multi-agent search system

The guide’s statement: coordinator who delegates to specialized sub-agents (web search, document analysis, synthesis, report generation). Output: full reports with quotes .

Domains summoned: 1 (heavily — this is the heart of the 27%), 2 (tool allocation), 5 (error propagation, provenance).

Hub-and-spoke architecture — the invariants:

  1. The coordinator owns all the communication inter-agents: decomposition, delegation, aggregation, error management. The subagents do not speak directly to each other (observability).
  2. Subagents have an isolated context : they do not inherit not of the coordinator’s history. Any necessary context must be explicitly included in their prompt . Recurring review question: a sub-agent produces an off-topic result → probable cause: the context of the previous phases was not transmitted to it.
  3. Parallelism : several calls Task in a single turn of the coordinator generate parallel sub-agents. THE allowedTools of the coordinator must include "Task".
  4. Prompts of the coordinator in objectives and quality criteria , not in step-by-step instructions (otherwise we lose the adaptivity which justifies multi-agents).

The three classic failures of scenario 3 — and their diagnosis:

Symptom Root cause Correction
The report only covers part of the subject Too narrow decomposition by the coordinator (he divided “creative industries” into 3 visual sub-themes) Review the coordinator's decomposition prompt — the sub-agents did what they were asked to do
Subagent timeout → entire workflow fails or generic status Poorly designed error propagation Structured error context : type of failure, query attempted, partial results, alternatives — the coordinator decides
Latency +40% due to back and forth verification The summary is passed back by the coordinator for each simple check Graduated Least Privilege : give synthesis a tool verify_fact limited for the 85% of simple cases, keep the coordinator path for the complex

Consensus and contradictions — the expected pattern: two credible sources give contradictory figures (40% vs 12%). The correct exam answer is not Never to choose heuristically, neither to escalate by blocking everything, nor to transmit without marking the conflict. It is : keep both values, explicitly annotate the conflict with source attribution, and let the coordinator reconcile . Add publication dates (a “contradiction” is often a temporal difference).

Quotes and provenance: the attribution is lost in the summary if we do not preserve the “affirmation → source” correspondences. Require structured output from subagents: assertion, URL/document name, citation, date. Refer to session 8 for full chain of provenance.

3.2 Scenario 7 — Conversational AI architectural models

The guide’s statement: multi-turn conversational systems — context window management, persistence of instructions across turns, memory strategies, design of tools for safe execution, ambiguous or contradictory inputs.

Domains summoned: 5 (heavily), 1, 4.

Structuring reminder #1 (the most tested): the API is stateless (stateless). Each query must return all necessary history. There is no "server-side session" in the Core Messages API. Stateful sessions are a application responsibility (or a feature of the SDK/Claude Code: --resume , fork_session ).

Memory strategies — the decision board:

Strategy Principle When Risk
Complete history Returning everything at every turn Short conversations Saturation + quadratic cost
Sliding window Keep only the last N turns Sessions where the distant past doesn't matter Loss of commitments made early
Progressive summary Summarizing ancient tricks Long sessions THE numerical values, dates, amounts dissolve into vague summaries
Hybrid (summary + recent verbatim + block of facts) Old summary + N verbatim rounds + persistent “case facts” excluding summary Serious production Implementation complexity

The “case facts” pattern is the reflex response: extract transactional facts (order number, amounts, decisions taken, commitments) in a persistent block never summarized , injected at each turn. This is the #1 risk avoidance of the progressive summary.

Persistence of instructions: the instructions of the system prompt are diluted in very long conversations (“lost in the middle” effect: the model handles the beginning and the end well, less the middle). Parades: reinject critical constraints near the end of the context, structure with XML tags (XML = eXtensible Markup Language) separating system / data / instructions.

Ambiguous or contradictory entries: ask for clarification when the ambiguity relates to an irreversible action; for multi-aspect requests, explicitly break it down into separate elements and handle them one by one.

Block 4 — Scenarios 2, 4 & 5: Claude Code (25 min)

This is the densest block: three scenarios for Domain 3 (20%) plus part of Domain 1. Announce the structure: configuration (S2) → productivity (S4) → CI/CD (S5).

4.1 Scenario 2 — Code generation with Claude Code

The four configuration mechanisms — hierarchy to know by heart:

Mechanism Location Scope Shared via VCS (VCS = Version Control System)?
CLAUDE.md user ~/.claude/CLAUDE.md All sessions of this user No
CLAUDE.md project CLAUDE.md root or .claude/CLAUDE.md The whole team on this repository Yes
CLAUDE.md directory project subdirectory Files in this subtree Yes
Targeted rules .claude/rules/*.md with frontmatter paths: (global patterns) Loaded uniquely by editing corresponding files Yes

The classic examination diagnosis: “a new member of the team does not have the conventions” → they are at the level user instead of the level project . Variant: “testing conventions must apply to files **/*.test.tsx scattered everywhere » → .claude/rules/ with glob pattern, not A CLAUDE.md by directory (the files concerned are in too many directories), not everything in the CLAUDE.md root (constant context load).

Modularization: syntax @path (@./standards/coding-style.md ) to include external files; .claude/rules/ themes (testing.md, api-conventions.md) rather than a CLAUDE.md monolithic.

Slash Commands and Skills:

Planning mode vs. direct execution — the decision rule:

4.2 Scenario 4 — Developer productivity tools

Integrated tools — who does what (almost guaranteed exam question):

Tool Use Do not confuse with
grep Search in the content files (function names, error messages, imports) Glob (filenames)
Globe Find files by name/extension patterns Grep (content)
Read/Write Read/write whole file Edit (targeted modification)
Edit Precise change by text match unique ; if not unique → fail → fallback Read + Write
bash Shell commands (tests, build, git)

The exploration pattern of an unknown code base: Grep entry points → Read to trace flows → build understanding incrementally . On exam: “the agent must understand how function X is used through wrapper modules” → Grep on the name, then Read call sites.

MCP in Claude Code:

4.3 Scenario 5 — Claude Code for continuous integration

Headless mode (headless = headless, non-interactive) — the exact flags:

# Le pipeline reste bloqué ? Il manque -p (--print) :
claude -p "Analyse cette pull request pour les problèmes de sécurité"

# Sortie structurée exploitable par le pipeline :
claude -p "Revue de sécurité de ce diff" \
  --output-format json \
  --json-schema review_schema.json

Recurring distractors: CLAUDE_HEADLESS=true , --batch , stdin redirection — Non-existent features or workarounds . The documented answer is -p / --print ⚠.

Architecture of an automated PR review (PR = Pull Request, merge request):

  1. Independent body for the review : the session which generated the code is less effective in revising it (it keeps its reasoning context and does not question its own decisions). Revise = new instance without generation context.
  2. CLAUDE.md as CI context : testing standards, review criteria, available fixtures — this is what makes the review relevant to THIS project.
  3. Minimize false positives : criteria explicit and categorical (“report: bugs, security; ignore: minor style”) rather than “be more conservative” (ineffective generic guidance). High false positive rates in one category undermine confidence in all others → temporarily disable noisy categories.
  4. Re-execution after new commits : include previous review results to only report new/uncorrected.
  5. Great PR : passes per file + inter-file integration pass (dilution of attention in a single pass — see session 8). The distractor: “take a model with a larger context window” — a larger window does not correct the quality of attention.

Batch vs real time in CI: verification blocking before merge → Synchronous API (developers wait); report of night (technical debt, weekly audit) → Message Batches API (−50% ⚠, processing window up to 24 hours ⚠, no latency SLA — SLA = Service Level Agreement, service level commitment). The Batch API does not support multi-turn tool calls in a query ⚠; correlation by custom_id ; in case of partial failure, resubmit only the failed elements.

Block 5 — Scenario 6: Extraction of structured data (15 min)

The guide’s statement: extract information from unstructured documents (invoices, contracts, semi-structured documents), validate using JSON schemas (JSON = JavaScript Object Notation), maintain high precision, manage edge cases.

Domains summoned: 4 (heavily), 5 (trusted calibration, human supervision).

5.1 The Reliable Extraction Stack — From Bottom to Top

  1. tool_use + JSON schema = the most reliable way to guarantee output conforming to the schematic. Eliminate errors from syntax JSON. With tool_choice: "any" or forced, the model cannot respond in free text.
  2. But the schema does not prevent semantic errors : totals that do not reconcile, value in the wrong field. Certain point of review: “we have a strict schema, why are there still errors?” » → the diagram validates the shape , not the sense .
  3. Design of the scheme against hallucination: fields optional/nullable when the source may not contain the information (a required field forces the model to invent); enums with "other"/"unclear" + detail field for extensibility.
  4. Semantic self-check: extract at once calculated_total (sum of rows) and stated_total (total displayed) to detect differences by programmatic comparison.
  5. Retry with error feedback: return the original document + the incorrect extraction + the precise validation errors . Absolute limit: retry is ineffective if the information is absent from the source (it is in an external document) — detect it and route through human review, not loop.
  6. Few-shot for ambiguous cases: 2-4 examples showing correct extraction on documents different structures — the few-shot reduces extraction hallucinations.

5.2 Calibration and human supervision (Domain 5 in scenario 6)

The trap of aggregated metrics: “97% overall accuracy” can hide 60% on a minority document type or on a specific field. Exam Answers:

Expected production release sequence: sample → prompt iteration on the sample → stratified validation → calibrated confidence thresholds → automation with continuous sampling. Any exam option that automates 100% from “overall accuracy is good” is a distractor.

Block 6 — Exam simulator (20 min)

Unfolded:

  1. Open session web page, tab Exam simulator . 12 questions, 18 timed minutes (real pace: 90 seconds/question).
  2. Each participant works alone, under examination conditions (no documentation).
  3. At the end: individual score + collective correction of the 3 most failed questions (the page displays the room statistics if the trainer records the scores by show of hands).

During the correction, hammer out the method, not the answer: for each failed question, have people verbalize: “Which distractor family got me?” » (magic prompt / over-engineering / invented feature / symptom). The objective is for everyone to leave with the awareness of her dominant bias.

Time strategy to transmit:

Block 7 — Revision plan & closing (5 mins)

  1. Domain tracker (web page, dedicated tab): each participant self-evaluates on the skills of the 5 areas with a level of confidence (🟢 mastered / 🟡 to review / 🔴 gap). The page calculates a readiness score weighted by exam weights .
  2. Prioritization rule: first revise the 🔴 of the heavy domains (D1 27%, D3 20%, D4 20%), then the 🟡. A 🔴 in D1 statistically costs ~4 questions; a 🔴 in D5, ~1.5.
  3. Logistics : check the official Anthropic page ⚠ for up-to-date terms and conditions (registration, monitoring, language, ironing policy). Remember that all numbers this session are volatile .
  4. Exit ticket: each participant notes their weakest scenario and their revision window. Announce the exercises (S1 architecture design, S3 comparison of approaches, S6 evaluation grid) and session 10.

Certification pitfalls — trainer’s cheat sheet

To distribute or project at the end of the session. The most profitable “quirks”:

  1. Deterministic vs probabilistic : critical business rule (money, identity, compliance) → hooks/preconditions, never the prompt alone.
  2. Tool descriptions = selection mechanism n°1. First correction in case of bad routing, before few-shot or routing layer.
  3. Subagents = isolated context. Nothing is inherited; everything happens explicitly in the prompt.
  4. Decomposition too narrow : when coverage of a report is partial, suspect the coordinator, not the subagents.
  5. Structured error > generic status ; access failure ≠ valid empty result; never mask failure as success.
  6. -p / --print for headless; --output-format json + --json-schema for the IC. CLAUDE_HEADLESS , --batch : do not exist ⚠.
  7. .claude/commands/ = team (versioned); ~/.claude/commands/ = personal. Same logic for .mcp.json (project) vs ~/.claude.json (user).
  8. .claude/rules/ + glob patterns when conventions apply per file type across the entire code base.
  9. JSON schema = syntax guaranteed, semantics not guaranteed. Nullable field against hallucination; useless retry if the information is missing from the source.
  10. Batch : −50% ⚠, ≤ 24 h ⚠, no SLA → never for blocking; custom_id to correlate; resubmit only failures.
  11. Low self-review : independent body to review; passes per file + integration pass for large PRs; “larger context window” does not repair attention.
  12. Feeling ≠ complexity; self-rated confidence ≠ reliable. Escalation on explicit criteria; explicit human request = immediate escalation.
  13. Progressive summary dilutes the numbers → “case facts” block persisting outside the summary.
  14. Aggregated metrics lie → stratify by document type and field; confidence at field level.
  15. No penalty on the exam → respond to everything, always.

Appendix — Correspondence scenarios × domains

Scenario D1 (27%) D2 (18%) D3 (20%) D4 (20%) D5 (15%)
S1 Customer Support ●● ●● ● ●●
S2 Code generation ● ●●● ●
S3 Multi-agent search ●●● ● ●●
S4 Dev Productivity ● ●● ●● ●
S5 Claude CI/CD code ●●● ●● ●
S6 Structured extraction ● ●●● ●●
S7 Conversational AI ●● ● ● ●●●
S8 Agentic Tools ●● ●●● ● ●

(●●● = dominant domain, ●● = strongly present, ● = present)

Strategic reading: D1 and D3 are everywhere — impossible to be weak there. D2 focuses on S1/S4/S8. D4 is mainly played on S5/S6. D5 is diffuse but peaks on S7.