Français
Applied AI · Advanced 🔴 · Session 10
📝 Teacher's Guide
← Return to program 📄 Source .md

Trainer's Guide — Advanced Level, Session 10

« Mock exam & final project »

Program : Applied AI — Yann Isola Audience : Solutions architects preparing for certification Claude Certified Architect Duration : 2 h 00 (tight structure: 45 min mock exam + 45 min defenses + 30 min retrospective) Prerequisites: Sessions 1 to 9 of the advanced level — this session is the capstone (capstone = final synthesis project): no new notion, only validation and consolidation. Material : interactive web page (webpage/index.html — complete exam simulator 30 questions/45 min, preparation dashboard with radar of the 5 areas, map of the 10 sessions), quiz/quiz.md (paper version of the mock exam with commented corrections), exercises/exercises.md (grid /100 of the final project + 3 warm-up challenges), visible stopwatch, projector, an evaluation grid printed per defense.

Session positioning

This is the last session of the advanced course and it has three distinct functions that should not be allowed to mix:

  1. Certify the preparation — the mock exam in real conditions gives everyone an honest measurement: ready, almost ready, or not ready. The role of the trainer is to make this measurement exploitable (diagnosis by area, revision plan), no reassurance.
  2. Bringing the final project to fruition — each participant designed along the way the complete architecture of a real AI system. Today he defends in front of the group. This is the exercise closest to the real profession: convincing peers that an architecture stands up.
  3. Close the loop — the retrospective links the 10 sessions into a coherent system and traces the rest: exam reservation, monitoring, continued practice.

⚠ Agreement: all figures marked ⚠ (exam format: 60 questions, 90 minutes, threshold 720/1000 ≈ 72%; weightings: 27/18/20/20/15%; prices, cache times, CLI flags) are volatile . Last reflex to install before leaving: we check the official source before reserving, before quantifying, before asserting.

Timed plan

Block Duration Content
0. Opening & examination instructions 5 mins Rules of the mock exam, reminder of the time management method
1. MAKE EXAM 45 mins 30 timed questions, real conditions, via the web simulator (or paper)
2. Flash Fix & diagnosis 10 mins Score by domain, radar reading, identification of red zones
3. FINAL PROJECT DEFENSES 45 mins 5 to 7 defenses of 6 min (4 min presentation + 2 min questions), cross-evaluation
4. Retrospective of the journey 12 mins Map of the 10 sessions, the 10 architect reflexes, what changes in production
5. Readiness self-assessment & plan 8 mins Preparation dashboard, individual decision: reserve or revise
6. Closing 5 mins Certification logistics, monitoring resources, final words

Time arbitration: if the group exceeds 7 participants, reduce the defenses to 5 minutes (3 + 2) and announce the constraint at the start. Born Never cut back on the 45 minutes of examination: the diagnostic value requires real conditions.

Block 0 — Opening & examination instructions (5 min)

Tagline message: “Ten sessions. Today, we don’t add anything – we check that everything fits. First alone faced with 30 questions, like on the big day. Then standing up, defending your architecture in front of your peers. At the end of the session, everyone knows whether they are reserving their exam or what they are reviewing first. »

Mock exam instructions (project)

  1. 30 questions, 45 minutes — same pace as the real exam: 90 seconds/question on average ⚠.
  2. No documentation, no AI — on D-day you will be alone.
  3. Reply to all — no penalty for error ⚠: an empty box is a point offered at random that is refused.
  4. Two-pass method : first quick pass (answer everything that is certain, mark the doubtful ones), second pass on the marked ones. The web simulator has a “mark for review” button that replicates this mechanism.
  5. MCQ reading grid (reminder of session 9): identify the root cause of the problem described → look for the answer least effort who processes it → check if the statement requires a guarantee (programmatic mechanism) or accepts a order (instruction to model). Three out of four distractors fall at this grid.

Logistics : each participant opens webpage/index.html , “Mock exam” tab, and starts the timer at the same time as everyone else. If there are no machines, distribute the paper quiz (without the corrections section!) and time it on the board.

Block 1 — Mock exam (45 min)

The trainer does not circulate And does not answer any substantive questions — real conditions. Three interventions authorized only:

During the exam, the trainer prepares block 3: order of presentation of defenses (drawing or volunteering noted on the board), evaluation grids distributed face down.

Distribution of questions (for trainer information)

Domain Weight exam ⚠ Questions Course sessions covered
D1 Architecture & orchestration 27 % Q1–Q8 S3 (SDK Agent), S4 (multi-agent)
D2 Tools & MCP 18 % Q9–Q13 S2 (tool use), S5 (MCP)
D3 Claude Code 20 % Q14–Q19 S6 (Claude Code & CI/CD)
D4 Prompt engineering & structured output 20 % Q20–Q25 S1 (API), S7 (prompt engineering)
D5 Background & reliability 15 % Q26–Q30 S8 (context, reliability, provenance)

Block 2 — Quick fix & diagnosis (10 mins)

The simulator automatically corrects and displays: overall score, score by domain , and a 5-axis radar. On paper: self-correction with the quick grid quiz.md (2 min), calculation by domain by hand.

Do NOT correct the 30 questions one by one — there is no time and the review mode of the simulator (detailed explanation by question) is made for individual post-session work.

Instead, correct 3 strategically chosen questions:

  1. The most unsuccessful of the group (the simulator does not report group stats: ask by show of hands on 4–5 questions that you suspect — typically Q20 prefill, Q27 cache, Q8 workflow/agent).
  2. A “deposit vs. guarantee” question (Q18 or Q28) — this is THE distinction that the exam tests under ten disguises.
  3. A “least effort” question (Q9 or Q4) — the distractor “switch to a more powerful model” or “deploy a complete agent” traps those who do not have the reflex of architectural economy.

Reading the score (project the interpretation scale of quiz.md ) :

Point of emphasis: a domain < 60% is a red zone even with a good overall score . The exam weights: D1 at 27% collapsed cannot be made up for by D5 at 15% perfect.

Block 3 — Final project defenses (45 min)

The format

Each participant presents the complete architecture of a real AI system designed throughout the course (instructions given in session 4, marked out since then). Strict format:

The evaluation grid (/100 — full details in exercises/exercises.md )

Criteria Points
1. Problem & business framing 15
2. Architecture & justification of choices 30
3. Tool design & integrations 15
4. Reliability, security & human supervision 20
5. Costs, evaluation & exploitation 10
6. Clarity of the defense 10

Cross-assessment: each participant notes two peers on the grid (assigned by the trainer to cover everyone), the trainer scores everyone. The indicative score = average. The objective is not the grade: it is to force everyone to listen as an evaluator , that is to say as an architect who challenges.

The deciding questions (trainer's bank)

Ask at least one question of this caliber per defense:

  1. “Your orchestrator breaks down at 3 a.m. What does the user see, and what does your system do on its own? »
  2. “Why an agent here, and not a fixed-step workflow? What is unpredictable in the way? »
  3. “Show me the exact point where a human approves. Why there and not elsewhere? »
  4. “This system prompt makes 2,000 tokens and leaves on each request. How much does it cost per month at your volume? Did you plan for the prompt cache? »
  5. “How will you know if the quality has gone down? Before What are users complaining about? »
  6. “What’s the worst thing a malicious user can make your system say or do?” »
  7. “If Anthropic changes the behavior of [volatile feature used], how does your system detect it? »

What we want to hear: answers which distinguish deposit/guarantee, which quantify, which admit the limits (“this case is not covered, here is why it is acceptable”). Red flag: “the model is intelligent, it will manage” — systematically sanction this response in the grid (criterion 4).

Group management

Block 4 — Retrospective of the course (12 min)

Use the “Course” tab of the web page: interactive map of the 10 sessions with their key concepts and connections.

The thread of the retrospective (unroll orally, supported by the map)

Session What we built The reflex that remains
S1 — Claude API in depth Raw calls, settings, streaming, vision The API is stateless; everything the model “knows” you put into context.
S2 — Advanced tool use Schemas, call orchestration, tool errors The tool description is a prompt. Tool error is reasoning data.
S3 — Agent SDK Agentic loops, sessions, state Workflow if the path is known, agent if it depends on the discoveries.
S4 — Multi-agent architecture Orchestrator, subagents, consensus, routing We cut to isolate contexts, not to look pretty on a diagram.
S5 — MCP in depth Servers, tools/resources/prompts, transports MCP standardizes the connection; the quality of the descriptions remains your job.
S6 — Claude Code & CI/CD CLAUDE.md, slash commands, headless, permissions CLAUDE.md is an instruction; settings.json is a guarantee.
S7 — Prompt engineering advanced CoT, few-shot, prefill, chains, evals A prompt is a software component: versioned, tested, measured.
S8 — Context, reliability & provenance Compaction, hiding, hallucinations, traceability Context is a managed resource. Confidence is journalized.
S9 — Certification scenarios Reading grid, 8 scenarios, revision plan Think like the questioner: root cause, least effort, guaranteed.
S10 — Capstone Mock exam, project defended An architecture that cannot be defended in 4 minutes is not understood.

The 3 meta-lessons (to be formulated by the group before projecting them)

Ask: “If you had to retain only three transversal principles from the entire journey? » Allow 2 minutes of discussion, then project:

  1. Deposit ≠ guaranteed. A prompt request; the code requires. Anything critical (permissions, schema validation, human approval, citation checking) must be enforced programmatically.
  2. The slightest architectural effort. A call before a channel; a string before an agent; an agent before multi-agents. Each level is paid for in cost, latency and debugging - we only go up if the level below has proven its insufficiency.
  3. Measure before you believe. Automated evaluations, golden sets, scores by domain, provenance logs: intuition initiates, measurement decides. This applies to the prompts as well as to your own preparation for the exam.

Block 5 — Readiness self-assessment & individual plan (8 min)

Everyone opens the “Preparation” tab of the web page: self-assessment on the 5 areas (5 statements per area, scale “I know how to do it / to consolidate / to repeat”), crossed with the score of the mock exam → preparation radar + recommendation.

Order : complete honestly, alone, in 5 min. The page generates a revision plan prioritized by exam weighting (a weak domain at 27% comes before a weak domain at 15%).

Individual decision to be made in writing before leaving (one line):

The trainer notes these commitments – it is the only “evaluation” of the block, and it conditions the post-training follow-up.

Block 6 — Closing (5 min)

Certification logistics (project, everything is volatile ⚠)

Continue after the route

  1. Practical > reading — a personal project with real API calls per week maintains the level; reading alone makes it decrease.
  2. Structured monitoring — Anthropic API changelog, Claude Code release notes, MCP specification: three sources, one monthly review. Everything with ⚠ in your notes comes from there.
  3. The community — share your architectures and challenge them: today’s defense was training for a permanent practice of the profession.

Final word (suggestion)

“You entered this journey knowing how to use a model. You come out knowing how to design systems : cut out, tooled, measured, governed. Certification will validate what you know — but what you defended today in front of the group, no one needs to certify: it was already the work of an architect. Go book that exam. »

Trainer Appendices

A. Mock exam correspondence scale → real exam

The blank has 30 questions where the exam has 60 ⚠: the variance is higher. Cautious rule: consider the interval score ± 2 questions before concluding. A 23/30 is “21 to 25”, therefore straddling the threshold → recommend consolidation week.

B. Last minute traps to remember if time permits

  1. “ALL choices seem true” questions → reread the question: it often asks first gesture, the more effective, or the lever main — a superlative changes the answer.
  2. The distractor “temperature 0 suppresses hallucinations” (false: reproducible ≠ true).
  3. The distractor “--dangerously-skip-permissions in CI” (never the right answer in an uncontrolled automated context).
  4. The distractor “the prompt system guarantees” (instruction ≠ guaranteed, always).
  5. The exact numbers (cache times, prices, limits): the exam tests the orders of magnitude and mechanisms , not the tariff gazette — but check the documentation the week of the exam ⚠.

C. If a participant fails badly in the blank (< 15/30)

Take 3 minutes aside after the session. Typical diagnosis: (a) lack of real API practice → prescribe practical exercises from sessions 1–3 with real calls; (b) reading statements too quickly → prescribing the two-passage method with 2 min/question in training; © confusion of domains D1/D2 → redo session 4 then 5, in that order. Never let someone go without a concrete plan.