Français
Applied AI · Intermediate 🟡 · Session 9
📝 Teacher's Guide
← Return to program 📄 Source .md

Trainer's Guide — Session 9: Building AI Products

Program : Applied AI — Intermediate level Trainer: Yann Isola Duration : 2 hours (120 minutes) Source module: Module 7

🎯 Educational objectives

At the end of this session, participants will know:

  1. Distinguish thinking produces technical thinking — starting from the problem, not the model
  2. Design a user experience adapted to the uncertainty inherent in AI
  3. Position a product on the “human in the loop” spectrum
  4. Assess When AI brings value — and when it doesn’t
  5. Measure an AI product with the right metrics (completion, time saved, trust, escalation)
  6. Calculate a cost-benefit ratio and a breakeven point
  7. Decide between buying (API), refining (fine-tuning) or building (from scratch)
  8. Integrate the principles of responsible AI by design

📋 Prerequisites and materials

Element Detail
Participant prerequisites Sessions 1–8 (LLM basics, prompting, RAG, agents, assessment)
Training material Slides (slides/slides.md ), interactive page (webpage/index.html ) open in a browser
Participant material Laptop, printed or accessible exercises, calculator (or spreadsheet)
Room Video projector, whiteboard, arrangement in islands of 3–4 for workshops

Logistics tip: the interactive page works offline — distribute it by USB key or intranet if the Wi-Fi is capricious.

⏱ Timed course (120 min)

Block 1 — Opening and product thinking (0:00 → 0:20, 20 min)

Time Activity Trainer notes
0:00–0:05 Home + hook Flash question: “Who has ever used a frustrating AI product?” For what ? » Collect 3–4 answers on the board. They will be used again throughout the session.
0:05–0:15 Product thinking vs. technical thinking (slides 3–6) The key message: “What problem am I solving?” » Before “Which model should I use? ». Case in point: a startup building a sophisticated RAG (Retrieval-Augmented Generation) chatbot… when users just wanted a quick search bar. Insist: the most impressive technology is not the most useful solution.
0:15–0:20 Mini-debate “Name a product where AI is invisible but useful” (e.g. anti-spam sorting, suggested replies in Gmail, automatic correction). Contrast with products where AI is put forward as a marketing argument with no real value.

Common trap: technical participants will want to talk about architecture. Reframe gently: “today, we wear the product hat, not the engineer hat”.

Block 2 — UX of uncertainty and human in the loop (0:20 → 0:45, 25 min)

Time Activity Trainer notes
0:20–0:32 UX (User Experience) for AI products (slides 7–10) Three pillars: manage uncertainty (the AI ​​is wrong — the product must predict it), trust indicators (show when AI is safe or not), graceful degradation (what happens when the AI ​​fails? The product remains usable). Examples: DeepL displays translation alternatives; GitHub Copilot offers without imposing; an OCR (Optical Character Recognition) highlights low confidence fields in orange for human proofreading.
0:32–0:42 The human spectrum in the loop (slides 11–13) Draw the spectrum on the board: fully automated → human approval → human control (override) → fully manual . Have participants place cases: spam filtering (auto), assisted medical diagnosis (approval), content moderation (override), drafting a contract (manual with assistance). Golden rule: the higher the cost of an error, the closer the human must be to the loop.
0:42–0:45 Exit ticket n°1 (quick oral) “For an automatic bank transfer detected as fraudulent: what level of human in the loop? Justify. » (Expected response: human approval — high error cost in both directions: blocking a real customer or letting fraud go through.)

Block 3 — When AI has value + metrics (0:45 → 1:10, 25 min)

Time Activity Trainer notes
0:45–0:55 When AI adds value (slides 14–16) The AI ​​value triangle: high volume + tolerance for imperfection + clear feedback loop . Positive examples: sorting 10,000 support tickets/day, meeting summaries, first draft of marketing content. Counterexamples: payroll calculation (zero tolerance for error), task executed twice a year (volume too low), final legal decision (unacceptable imperfection). Do the opposite exercise: “give me a task from your job and let’s test it against the triangle”.
0:55–1:07 AI product metrics (slides 17–19) Four key metrics: task completion rate (does the user finish what he started?), time saved (measured, not declared), user trust score (surveys, reuse rate), escalation rate (how many times does the human have to take control?). Then A/B testing (comparison of two variants on distinct populations) applied to AI: prompt variants, model routing (small model for simple cases, large for complex ones), feature flags (software switches to activate a function for a subgroup). Numerical example: variant A of the prompt → 72% completion, variant B → 81% → we deploy B progressively via feature flag.
1:07–1:10 Exit ticket n°2 (written, 3 min) “Your support chatbot has an escalation rate that goes from 20% to 35% after an update. List two possible causes and one action. »

Recommended break: 5 minutes at 1:10 if the group gets tired (deduct on block 4, which has margin).

Block 4 — Cost-benefit and build/buy/fine-tune (1:10 → 1:35, 25 min)

Time Activity Trainer notes
1:10–1:22 Cost-benefit analysis (slides 20–22) Cost side: API (Application Programming Interface) calls, infrastructure, maintenance, human supervision. Value side: time saved × hourly cost, additional income, quality gains. Live demo : open the interactive page, “ROI Calculator” (Return On Investment) tab. Enter a case: manual processing of a ticket = 12 min at €30/h; AI cost ≈ €0.08/ticket ⚠ (API prices change quickly); volume 5,000 tickets/month; AI accuracy 85% (15% scaled to human). Show the balance point live.
1:22–1:32 The build/buy/fine-tune decision tree (slides 23–25) Three branches: buy (off-the-shelf API) — reasonable defect for 80% of cases: fast, little maintenance; fine-tuning — when the domain is specific, the volume justifies the cost, and the training data exists; build (from scratch) — extremely rare: sovereignty constraints, cases where the model IS the product, substantial budgets. Demo: “Decision tree” tab of the interactive page. Have the group answer questions for 2 scenarios: (a) internal email classification, (b) proprietary fraud detection model.
1:32–1:35 Exit ticket n°3 (oral) “An SME (Small and Medium Enterprise) wants a quote writing assistant. Build, buy or fine-tune? » (Expected: buy — API + prompting is enough; fine-tuning is only justified with proven volume and specificity.)

Block 5 — Trust, responsible AI and case studies (1:35 → 1:55, 20 min)

Time Activity Trainer notes
1:35–1:43 Build Trust (slides 26–27) Three levers: explain what the AI ​​did (not how it works - the user doesn't care about transformers, he wants to know "I summarized these 3 documents"); show sources (clickable quotes in a RAG); allow corrections (“this is wrong” button which powers the improvement). Anti-example: an assistant who asserts without ever citing → confidence which erodes with the first error detected.
1:43–1:50 Responsible AI in products (slide 28) Four concrete projects: bias detection (test the product on subpopulations: does CV sorting score differently depending on the first name?), content filtering (inputs AND outputs), user consent (does the user know that he is talking to an AI? GDPR - General Data Protection Regulation), data privacy (what data goes to the API? Are they used for training?). Mention the European AI Act ⚠ (application schedule currently evolving).
1:50–1:55 Flash Case Studies (slides 29–30) Three cases in 90 seconds each: (1) Customer Support Automation — auto triage + suggested response + human escalation, metrics: first contact resolution rate, escalation. (2) Content generation pipeline — human brief → AI draft → human review → publication; the AI ​​makes the first draft, never the validation. (3) Internal knowledge base — RAG on corporate documentation with mandatory citations and report button. Connect each case to the value triangle and the human in the loop spectrum.

Block 6 — Summary and closing (1:55 → 2:00, 5 min)

Time Activity Trainer notes
1:55–1:58 Summary in 4 sentences 1. The problem before the model. 2. Uncertainty is designed, it cannot be hidden. 3. Measure: completion, time, confidence, escalation. 4. Buy by default, fine-tune if justified, build almost never.
1:58–2:00 Exit tickets n°4 and n°5 + announcements Distribute the last two exit tickets (below). Announce the exercises to be completed and the next session.

🎫 The 5 exit tickets (summary)

  1. (Oral, block 2) Suspected fraudulent transfer: what level of human in the loop and why?
  2. (Writing, block 3) Escalation rate 20% → 35% after update: two possible causes, one action.
  3. (Oral, block 4) Quote assistant for an SME: build, buy or fine-tune?
  4. (Written, closing) “Cite the three conditions of the AI ​​value triangle and give a counter-example from your profession. »
  5. (Written, closing) “An AI product displays an answer without a source or confidence indicator. Propose two concrete UX improvements and say which metric they would improve. »

Quick fix: tickets 4 and 5 are corrected in 30 seconds each. A failed ticket 4 = review block 3 at the start of session 10.

⚠ Points of vigilance trainer

📚 To go further (to share after the session)