# Trainer's Guide — Session 9: Building AI Products

**Program:** Applied AI — Intermediate level
**Trainer:** Yann Isola
**Duration:** 2 hours (120 minutes)
**Source module:** Module 7

---

## 🎯 Educational objectives

At the end of this session, participants will know:

1. **Distinguish** product thinking from technical thinking — start from the problem, not the model
2. **Design** a user experience adapted to the uncertainty inherent in AI
3. **Position** a product on the “human in the loop” spectrum
4. **Assess** when AI brings value — and when it doesn't
5. **Measure** an AI product with the right metrics (completion, time saved, trust, escalation)
6. **Calculate** a cost-benefit ratio and a breakeven point
7. **Decide** between buying (API), refining (fine-tuning) or building (from scratch)
8. **Integrate** responsible AI principles by design

---

## 📋 Prerequisites and materials

| Element | Detail |
|---|---|
| Participant prerequisites | Sessions 1–8 (LLM basics, prompting, RAG, agents, assessment) |
| Training material | Slides (`slides/slides.md`), interactive page (`webpage/index.html`) opened in a browser |
| Participant materials | Laptop, printed or accessible exercises, calculator (or spreadsheet) |
| Room | Video projector, whiteboard, arrangement in islands of 3–4 for workshops |

**Logistics tip:** the interactive page works **offline** — distribute it via USB or intranet if Wi-Fi is finicky.

---

## ⏱ Timed sequence (120 min)

### Block 1 — Openness and product thinking (0:00 → 0:20, 20 min)

| Time | Activity | Trainer notes |
|---|---|---|
| 0:00–0:05 | Home + teaser | Flash question: “Who has ever used a frustrating AI product?” For what ? » Collect 3–4 answers on the board. They will be used again throughout the session. |
| 0:05–0:15 | **Product thinking vs technical thinking** (slides 3–6) | The key message: *“What problem am I solving? »* before *« Which model should I use? »*. Case in point: a startup building a sophisticated RAG (Retrieval-Augmented Generation) chatbot… when users just wanted a quick search bar. Emphasize: the most impressive technology is not the most useful solution. |
| 0:15–0:20 | Mini-debate | “Name a product where AI is invisible but useful” (e.g. anti-spam sorting, suggested replies in Gmail, automatic correction). Contrast with products where AI is put forward as a marketing argument with no real value. |

**Common pitfall:** technical participants will want to talk about architecture. Reframe gently: “today, we wear the product hat, not the engineer hat”.

---

### Block 2 — UX of uncertainty and human in the loop (0:20 → 0:45, 25 min)

| Time | Activity | Trainer notes |
|---|---|---|
| 0:20–0:32 | **UX (User Experience) for AI Products** (slides 7–10) | Three pillars: **managing uncertainty** (the AI ​​makes a mistake — the product must predict it), **confidence indicators** (showing when the AI ​​is safe or not), **graceful degradation** (what happens when the AI ​​fails? The product remains usable).Examples: DeepL displays translation alternatives; GitHub Copilot offers without imposing; an OCR (Optical Character Recognition) highlights low confidence fields in orange for human proofreading. |
| 0:32–0:42 | **The human specter in the loop** (slides 11–13) | Draw the spectrum on the board: **fully automated → human approval → human control (override) → fully manual**. Have participants place cases: spam filtering (auto), assisted medical diagnosis (approval), content moderation (override), drafting a contract (manual with assistance). Golden rule: the higher the cost of an error, the closer the human should be to the loop. |
| 0:42–0:45 | Exit ticket n°1 (quick oral) | “For an automatic bank transfer detected as fraudulent: what level of human in the loop? Justify. » (Expected answer: human approval — high error cost in both directions: blocking a real customer or letting a fraud pass.) |

---

### Block 3 — When AI has value + metrics (0:45 → 1:10, 25 min)

| Time | Activity | Trainer notes |
|---|---|---|
| 0:45–0:55 | **When AI adds value** (slides 14–16) | The AI ​​Value Triangle: **high volume** + **tolerance for imperfection** + **clear feedback loop**. Positive examples: sorting 10,000 support tickets/day, meeting summaries, first draft of marketing content. Counterexamples: payroll calculation (zero tolerance for error), task executed twice a year (volume too low), final legal decision (unacceptable imperfection). Do the opposite exercise: “give me a task from your job and let’s test it against the triangle”. |
| 0:55–1:07 | **AI Product Metrics** (slides 17–19) | Four key metrics: **task completion rate** (does the user finish what they started?), **time saved** (measured, not declared), **user confidence score** (surveys, reuse rate), **escalation rate** (how many times does the human have to take control?). Then **A/B tests** (comparison of two variants on distinct populations) applied to AI: prompt variants, model routing (small model for simple cases, large for complex ones), feature flags (software switches to activate a function for a subgroup). Numerical example: variant A of the prompt → 72% completion, variant B → 81% → we deploy B progressively via feature flag. |
| 1:07–1:10 | Exit ticket n°2 (written, 3 min) | “Your support chatbot has an escalation rate that goes from 20% to 35% after an update. List two possible causes and one action. » |

**Recommended break: 5 minutes at 1:10 if the group gets tired** (deduct from block 4, which has some margin).

---

### Block 4 — Cost-benefit and build/buy/fine-tune (1:10 → 1:35, 25 min)

| Time | Activity | Trainer notes |
|---|---|---|
| 1:10–1:22 | **Cost-benefit analysis** (slides 20–22) | Cost side: API (Application Programming Interface) calls, infrastructure, maintenance, human supervision. Value side: time saved × hourly cost, additional income, quality gains. **Live demo**: open the interactive page, “ROI Calculator” tab (Return On Investment).Enter a case: manual processing of a ticket = 12 min at €30/h; AI cost ≈ €0.08/ticket ⚠ (API prices change quickly); volume 5,000 tickets/month; AI accuracy 85% (15% scaled to human). Show balance point live. |
| 1:22–1:32 | **The build / buy / fine-tune decision tree** (slides 23–25) | Three branches: **buy (off-the-shelf API)** — reasonable default for 80% of cases: fast, low maintenance; **fine-tuning** — when the domain is specific, the volume justifies the cost, and the training data exists; **build (from scratch)** — extremely rare: sovereignty constraints, cases where the model IS the product, substantial budgets. Demo: “Decision tree” tab of the interactive page. Have the group answer questions for 2 scenarios: (a) internal email classification, (b) proprietary fraud detection model. |
| 1:32–1:35 | Exit ticket n°3 (oral) | “An SME (Small and Medium Enterprise) wants a quote writing assistant. Build, buy or fine-tune? » (Expected: buy — API + prompting is enough; fine-tuning is only justified with proven volume and specificity.) |

---

### Block 5 — Trust, responsible AI and case studies (1:35 → 1:55, 20 min)

| Time | Activity | Trainer notes |
|---|---|---|
| 1:35–1:43 | **Building Trust** (slides 26–27) | Three levers: **explain what the AI ​​did** (not how it works - the user doesn't care about transformers, he wants to know "I summarized these 3 documents"); **show sources** (clickable citations in a RAG); **allow corrections** (“this is wrong” button which powers the improvement). Anti-example: an assistant who asserts without ever citing → confidence which erodes with the first error detected. |
| 1:43–1:50 | **Responsible AI in products** (slide 28) | Four concrete projects: **bias detection** (testing the product on subpopulations: does CV sorting score differently depending on the first name?), **content filtering** (entries AND exits), **user consent** (does the user know that he is talking to an AI? GDPR — General Data Protection Regulation), **data confidentiality** (what data goes to the API? Is it used for training?). Mention the European AI Act ⚠ (application schedule currently evolving). |
| 1:50–1:55 | **Flash case studies** (slides 29–30) | Three cases in 90 seconds each: **(1) Customer support automation** — auto triage + suggested response + human escalation, metrics: first contact resolution rate, escalation. **(2) Content generation pipeline** — human brief → AI draft → human review → publication; the AI ​​makes the first draft, never the validation. **(3) Internal Knowledge Base** — RAG on corporate documentation with required citations and report button. Connect each case to the value triangle and the human in the loop spectrum. |

---

### Block 6 — Summary and closing (1:55 → 2:00, 5 min)

| Time | Activity | Trainer notes |
|---|---|---|
| 1:55–1:58 | Summary in 4 sentences | 1. The problem before the model. 2. Uncertainty is designed, it cannot be hidden. 3. Measure: completion, time, confidence, escalation. 4. Buy by default, fine-tune if justified, build almost never. |
| 1:58–2:00 | Exit tickets n°4 and n°5 + announcements | Distribute the last two exit tickets (below).Announce the exercises to be completed and the next session. |

---

## 🎫 The 5 exit tickets (summary)

1. **(Oral, block 2)** Suspected fraudulent transfer: what level of human in the loop and why?
2. **(Written, block 3)** Escalation rate 20% → 35% after update: two possible causes, one action.
3. **(Oral, block 4)** Quote assistant for an SME: build, buy or fine-tune?
4. **(Writing, closing)** “Cite the three conditions of the AI ​​value triangle and give a counter-example from your profession. »
5. **(Writing, closing)** “An AI product displays an answer without a source or confidence indicator. Propose two concrete UX improvements and say which metric they would improve. »

**Quick fix:** Tickets 4 and 5 can be fixed in 30 seconds each. A failed ticket 4 = review block 3 at the start of session 10.

---

## ⚠ Trainer vigilance points

- **API cost figures are volatile ⚠** — announce orders of magnitude as “as I speak” and show where to check (vendor pricing pages).
- **Avoid the “AI will replace humans” debate** — refocus on the human in the loop spectrum: the question is not “to replace or not” but “where to place the cursor”.
- **Non-technical participants:** block 4 (cost-benefit) is their highlight — having them manipulate the calculator themselves.
- **Very technical participants:** positively frustrate them — ask them to defend a “buy” when they dream of “build”.
- **Tight timing on block 5** — if late, cut case study #2 (content pipeline) rather than the responsible AI, which is non-negotiable.

---

## 📚 To go further (to share after the session)

- The exercises (`exercises/exercises.md`) — the canvas produced can be done in pairs by session 10
- The quiz (`quiz/quiz.md`) — 10 questions, to do independently, correction at session opening 10
- The interactive page (`webpage/index.html`) — canvas, ROI calculator and decision tree reusable on their own projects