Français

Slides — Session 9: Building AI Products

Program: Applied AI — Intermediate level · Trainer: Yann Isola
28 slides · 2 hours · Palette: ink #1A2230, teal #0F7A6C, copper #B4612A

Slide 1 — Title

Build AI products

From model to product: think about the user, measure, decide

Applied AI — Session 9 · Yann Isola

Slide 2 — Session program

  1. 🧭 Product thinking vs technical thinking
  2. 🎨 UX of uncertainty & human in the loop
  3. 📐 When AI has value — and the metrics to prove it
  4. 💰 Cost-benefit & the build/buy/fine-tune tree
  5. 🤝 Trust, responsible AI, case studies

Slide 3 — The hook

“Who has ever used a frustrating AI product?”

(harvest on the board — we’ll come back to that)

Slide 4 — Two ways of thinking

Technical thinking Product thinking
“Which model should I use?” “What problem am I solving?”
“Grok 4.5 or Claude? RAG or fine tuning?” “For who?” What pain? What frequency?”
Part of the solution Part of the problem
Success = it works technically Success = user returns

Slide 5 — Anti-example from real life

The startup with the magnificent chatbot

  • 6 months of development: sophisticated RAG, query rewriting, reranking
  • User interviews after launch: “In fact, we just wanted a quick search with filters”
  • 💸 Solution in search of problem

Slide 6 — Invisible AI

The best AI products don't scream "AI!"

  • 📧 Anti-spam sorting: no one thinks about it, everyone uses it
  • ✍️ Answer suggestions, auto-correction
  • 🗺️ Estimated travel time

AI is a means, not an argument.

Slide 7 — Uncertainty is a design consideration

The AI is wrong. Always. Sometimes.

The product must be designed knowing that:

  • some answers will be wrong,
  • the user will not know which ones,
  • …unless the product helps him know it.

Slide 8 — Three pillars of AI UX

  1. Manage uncertainty — anticipate the error in the journey
  2. Trust Indicators — show when AI is safe… and when it is not
  3. Graceful degradation — when AI fails, the product remains usable

Slide 9 — Concrete examples

Product Good practice
DeepL Proposes translation alternatives — assumes non-uniqueness
GitHub Copilot Suggests without imposing — the human accepts or ignores
Invoice OCR Highlight low confidence fields in orange → targeted proofreading
Support Assistant “I’m not sure — do you want to talk to a counselor?”

Slide 10 — What NOT to do

  • ❌ Affirm with the same confidence the true and the false
  • ❌ Hide failure cases (“no result” silently)
  • ❌ Block the entire flow as soon as an element is uncertain
  • ❌ Drowning the user with incomprehensible technical scores

Slide 11 — The “human in the loop” spectrum

Fully            Human            Human oversight      Fully
automated   →   approval    →      (override)     →    manual
   🤖              🤖✋👤             🤖👁👤              👤(🤖 assists)

Human in the loop = human in the loop.

Slide 12 — Where to place the cursor?

Golden rule: the higher the cost of an error, the closer to humans.

Case Position Why
Spam filtering Automated Benign, reversible error
Content moderation Override Huge volume, but necessary recourse
Help with medical diagnosis Approval Major cost of error
Contract drafting Assisted manual Legal liability

Slide 13 — The cursor moves over time

  • Launch: human approval (we learn, we measure)
  • Established trust + stable metrics: shift towards override
  • Simple cases only: progressive automation

We automate on proof, not on promise.

Slide 14 — When AI adds value

The value triangle

        High volume
           ▲
          ╱ ╲
         ╱   ╲
        ╱  💎 ╲
       ╱───────╲
Tolerance of    Clear feedback
imperfection    loop

All three conditions together — not two out of three.

Slide 15 — The triangle in practice

Task Volume Tolerance Feedback Verdict
Sorting of 10,000 tickets/day ✅ ✅ ✅ (reclassifications) 💎 Ideal
Meeting summaries ✅ ✅ ⚠️ 👍 Good
Payroll calculation ✅ ❌ ✅ ❌ No
Annual shareholders report ❌ ❌ ❌ ❌ No

Slide 16 — The “two out of three” trap

  • Volume + tolerance, without feedback → the product stagnates, the quality drifts without being seen
  • Volume + feedback, without tolerance → each error costs too much (payroll, transfers, legal)
  • Tolerance + feedback, without volume → the game is not worth the effort (bi-annual task)

Slide 17 — The 4 key metrics of an AI product

Metric Question she answers
Task completion rate Does the user finish what they started?
Time saved Measured (timestamp), not reported
User Trust Score Surveys + reuse rate
Escalation rate How often do humans take control?

Slide 18 — Read the metrics together

An example that should worry:

  • Completion: 85% ✅
  • Time saved: +30% ✅
  • Climbing: 20% → 35% in 2 weeks 🔴

What happened?

Slide 19 — A/B testing for AI features

Three objects to test:

  1. Prompt variations — A: 72% completion, B: 81% → B wins
  2. Model routing — fast small model for simple cases, big model for complex ones
  3. Feature flags (software switches) — activate for 5% of users, observe, expand

Slide 20 — Cost-benefit: the two sides of the scale

💸 Costs 💎 Value created
API calls (per request) ⚠ Time saved × hourly cost
Infrastructure & accommodation Additional income
Maintenance & evolution of prompts Quality gains (fewer errors, fewer reworks)
Human supervision & escalations Customer satisfaction/retention

Slide 21 — Live demo: the ROI calculator

Case: customer support, 5,000 tickets/month

  • Manual processing: 12 min/ticket at €30/h
  • AI cost: ≈ €0.08/ticket ⚠ + infrastructure package
  • Accuracy: 85% (15% scaled)

→ Interactive page, “ROI Calculator” tab

Slide 22 — The calculator lesson

Accuracy dominates API cost

  • API cost ×3 → economy drops by a few%
  • Accuracy 85% → 65% → economy collapses (escalations = complete human labor)

Manage the escalation rate, not the price negotiation.

Slide 23 — The decision tree: build / buy / fine-tune

Need identified
   │
   ├─ Off-the-shelf API + good prompt is enough? ──── YES → 🛒 BUY (≈80% of cases)
   │                │ NO
   ├─ Specific domain + training data
   │  + volume that justifies the cost? ───────────── YES → 🔧 FINE-TUNE
   │                │ NO
   └─ Total sovereignty required / the model IS
      the product / substantial budget? ───────────── YES → 🏗 BUILD (rare)

Slide 24 — Buy / Fine-tune / Build: the comparison

🛒 Buy (API) 🔧 Fine-tune 🏗️ Build
Deadline Days Weeks Months/years
Initial cost Low Medium Very high ⚠
Skills Prompting, integration + data, evaluation + ML search
Maintenance Minimal Continue (re-workouts) Total
When ? Default (≈80%) Ceiling accuracy + volume + data Sovereignty / core business

Slide 25 — Exit ticket n°3

An SME wants a quote writing assistant.

Build, buy or fine-tune? Justify in one sentence.

Slide 26 — Building trust: 3 levers

  1. Explain what the AI did — not how it works
    “I summarized these 3 documents” > “transformer architecture with 70 billion parameters”
  2. Show sources — clickable citations, traceability
  3. Allow corrections — “this is wrong” button that powers improvement

Slide 27 — Trust is asymmetrical

  • 📈 It builds slowly: dozens of successful interactions
  • 📉 It destroys itself quickly: a serious error asserted with aplomb
  • 🔁 It can be repaired with transparency, not with promises

Design for the day the AI gets it wrong. That day will come.

Slide 28 — Responsible AI: 4 concrete projects

Construction site Test question
Bias detection Does the product rate differently depending on the first name on a CV?
Content filtering Inputs AND outputs — is it resistant to prompt injection?
Consent Does the user know that they are interacting with an AI?
Confidentiality What data goes to the API? Reused for training?

⚠ Regulatory framework: European AI Act (evolving application schedule) + GDPR.

Slide 29 — Three flash case studies

Case Editing Key Metrics
Customer Support Auto triage + suggested response + human escalation 1st contact resolution, escalation rate
Content Pipeline Human brief → AI draft → human review → publication Time saved, recovery rate
Internal knowledge base RAG + mandatory citations + report button Completion, confidence, reports

Slide 30 — Summary & next step

To remember

  1. 🧭 The problem before the model
  2. 🎨 Uncertainty is design, it cannot be hidden
  3. 📐 Value = volume + tolerance + feedback; measure: completion, time, confidence, escalation
  4. 💰 Buy by default, fine-tune on proof, build almost never
  5. 🤝 Trust = explain, source, correct — and responsibility by design

For session 10

  • Exit tickets n°4 and n°5 (now, 5 min)
  • Exercises: product canvas (possible pair), cost-benefit, responsible AI audit
  • Quiz of 10 questions in autonomy

Notes: Home. Announce the common thread: “today, we are not talking about how AI works, but about how we make it a product that people use, that they trust, and that brings in more than it costs.”

Notes: 5 blocks, 2 live demos on the interactive page (ROI calculator and decision tree). Warning: 5 exit tickets punctuate the session.

Notes: 3–4 answers max. Typically: chatbot going around in circles, false answers asserted with aplomb, no way to speak to a human. Keep these examples visible: they illustrate blocks 2 and 5.

Notes: RAG = Retrieval-Augmented Generation. Hammer: the technical question is legitimate, but it comes **second**. The most impressive technology is not the most useful.

Notes: Composite but representative case. The cost is not only financial: 6 months behind the real need. Question to the group: “What should have been done on day 1?” (Answer: talk to users before coding.)

Notes: Contrast with “AI-powered” products as a worthless marketing argument. Transition: “if AI is a means, we still need to manage its major flaw: it makes mistakes.”

Notes: This is THE difference with classic (deterministic) software. A “save” button works or crashes. An AI response may be *plausibly wrong*. The whole UX flows from that.

Notes: UX = User Experience. Examples next slide. “Graceful degradation”: if the AI ​​service goes down, the user should be able to continue manually — never a dead screen.

Notes: OCR = Optical Character Recognition. Common point: uncertainty is **visible and actionable**, never hidden. Question 2 of the quiz focuses on this.

Notes: The last point is subtle: displaying “confidence: 0.87” doesn't speak to anyone. Prefer simple codes: green / orange / “to check”. Take the frustrations collected slide 3 — most fit into these boxes.

Notes: Draw on the board. Clear definitions: *automated* = the AI ​​acts alone; *approval* = nothing is executed without prior validation; *override* = the AI ​​acts, the human monitors and can override; *manual* = human does, AI assists.

Notes: Participatory exercise: propose other cases (suspicious transfer, sorting of CVs, customer response) and have the group vote on the position. Exit ticket n°1 just after: fraudulent transfer → human approval.

Notes: Point often forgotten: the position is not fixed. The escalation rate (next block) is precisely the instrument that tells whether we can move the cursor.

Notes: Each condition: *volume* (the task comes up often enough to absorb the effort), *tolerance* (an error is corrected without disaster), *feedback* (we know how to measure whether the AI is doing well, and the signal fuels the improvement).

Notes: Reverse exercise with the group: “give a task from your job, let’s test it against the triangle”. 2–3 rounds. This is exercise 1.1 of the exercise folder — and exit ticket #4.

Notes: Emphasize the first case, the most sneaky: the product “works” at launch then silently deteriorates. Hence the metric block that follows.

Notes: The escalation rate is the canary in the mine: it often goes up **before** complaints arrive. “Time saved”: distrust of the declarative – users overestimate or underestimate; measure actual timestamps.

Notes: Have the group searched: update of the prompt/model which degraded a use case, new segment of users with different needs, drift of input data, etc. Action: compare the escalated cases before/after, rollback if necessary. This is exit ticket n°2 (written, 3 min).

Notes: A/B test = compare two variants on distinct populations. Methodological reminders: statistical significance (a difference over 40 users proves nothing), only one variable at a time, progressive deployment. Quiz question 6.

Notes: API = Application Programming Interface. ⚠ API pricing is changing quickly — give orders of magnitude “as I speak” and show official pricing pages. The most underestimated cost: **human supervision** — a product with 82% precision retains 18% manual processing.

Notes: ROI = Return On Investment. Live demo: enter values, show monthly economy and breakeven. Then play: “What happens if the accuracy drops to 60%?” → show that precision weighs more than API cost. This is the heart of exercise 2.

Notes: This is the expected conclusion of exercise 2.4. Product corollary: investing in improving precision (better prompts, better data) pays off more than changing supplier for 20% less.

Notes: Fine-tuning = refining an existing model on its own data. Build from scratch = build from scratch. Important reading order: we go down the tree, we don't jump floors. “Buy” is the default; the other two are justified by proof.

Notes: ML = Machine Learning. Demo: “Decision tree” tab of the interactive page — have the group respond for 2 scenarios: classification of internal emails (→ buy), very high volume proprietary fraud detection (→ fine-tune, or even build if the model is the competitive advantage).

Notes: SME = Small and Medium Enterprise. Expected response: **buy** — an API + a structured prompt with quote templates is enough; fine-tuning would only be justified with a high volume and specificity proven after testing. If someone defends fine-tune, ask: "with what data?" at what cost? to win what?”

Notes: Anti-example: the assistant who asserts without citing → confidence collapses at the **first** error detected, and does not return. User correction closes the feedback loop of the value triangle — it all comes together.

Notes: Link to the frustrations in slide 3. Transition to responsible AI: “user trust is one thing. Responsibility to those affected is another.”

Notes: GDPR = General Data Protection Regulation. CV = Curriculum Vitae. The first name test is real (studies on CV sorting bias). Exercise 3 is a full audit of one such case — announce it here.Non-negotiable even if the timing is tight.

Notes: 90 seconds per case. For each person, have the group name: position on the human in the loop spectrum, and validation of the value triangle. Common point of the three: **the AI makes the first draft, the human keeps the decision.

Notes: Distribute the last two exit tickets. Remember that the interactive page is reusable on their own projects — it is the tool, not just the course support. Fence.