# Quiz — Session 9: Building AI products **Program:** Applied AI — Intermediate level · **Trainer:** Yann Isola **10 multiple choice questions — only one correct answer per question** **Recommended duration: 15 minutes · Correction at login 10** --- ### Question 1 — Product thinking What is the **first question** to ask when considering an AI feature? - A. Which model performs the best in recent benchmarks? - B. What user problem does this feature solve? - C. Should we use RAG (Retrieval-Augmented Generation) or fine-tuning? - D. What is the cost per API (Application Programming Interface) call?
Response **B.** Product thinking starts from the problem, never from the technology. Questions A, C and D are legitimate, but they come later — otherwise we construct a solution in search of a problem.
--- ### Question 2 — UX of uncertainty An OCR (Optical Character Recognition) tool extracts invoice amounts. The **best** UX (User Experience) practice is: - A. Only display the amounts of which the AI is certain, hide the others - B. Display all amounts in the same way so as not to worry the user - C. Highlight low confidence amounts to invite targeted rereading - D. Block the entire document as soon as a field is uncertain
Response **C.** A good AI product makes uncertainty **visible and actionable**: the user concentrates their rereading where it is useful. A hides information, B deceives the user, D destroys the value of the product (brutal degradation instead of graceful).
--- ### Question 3 — Human in the loop On the “human in the loop” spectrum, the level **“human approval”** means: - A. The AI acts alone and the human can cancel afterwards - B. The AI proposes, but nothing is executed without prior human validation - C. The human does everything, the AI observes - D. AI and humans work in parallel on different tasks
Response **B.** Approval = validation **before** execution. Answer A describes the “override” level (posterior control). The higher the cost of making a mistake, the more one moves toward pre-approval, or even manual.
--- ### Question 4 — Value triangle Among these tasks, which **best** meets the three conditions of AI value (high volume, tolerance for imperfection, clear feedback loop)? - A. The monthly calculation of employee pay - B. Drafting the annual report intended for shareholders - C. Daily sorting of thousands of customer support tickets by category - D. The signing of a merger-acquisition contract
Response **C.** Huge volume ✅, a category error is easily corrected ✅, reclassification by agents provides a signal of improvement ✅. A fails on tolerance (zero admissible errors in payroll), B on volume (once/year), D on both.
--- ### Question 5 — AI product metrics The **escalation rate** of an AI assistant measures: - A. The percentage of responses generated in less than a second - B. The proportion of cases where a human must take control - C.The number of monthly active users - D. The average cost per request
Response **B.** Climbing is the transfer towards humans. This is a central metric: a rising escalation rate signals a deterioration in the perceived or actual quality of the product — often before other metrics move.
--- ### Question 6 — A/B testing You test two prompt variations on your assistant. Variant A obtains 72% task completion, variant B 81%. The **correct** approach is: - A. Deploy B immediately to 100% of users - B. Check statistical significance and volume, then deploy B progressively via a feature flag (software activation switch) - C. Keep A, because changing a prompt in production is risky - D. Merge the two prompts to get the best of both
Response **B.** A discrepancy may be due to chance if the sample is small; and even validated, a progressive deployment (feature flag) limits the risk. A skips the steps, C is immobility, D has no methodological basis.
--- ### Question 7 — Cost-benefit A manual task costs €40,000/month. The AI-assisted version costs €15,000/month (API + infrastructure + human supervision). The integration project cost €50,000. The **breakeven point** is reached in: - A. 1 month - B. 2 months - C. approximately 3.3 months - D. 50 months
Response **B.** Monthly savings = 40,000 − 15,000 = €25,000. Breakeven = 50,000 / 25,000 = **2 months**. Reflex to remember: always think about the **net** savings (including the residual human cost of supervision and escalation), not just the API cost.
--- ### Question 8 — Building trust For a user to trust a document research assistant, the **most effective** practice is: - A. Explain the model architecture (transformers, attention) in the online help - B. Show cited sources and allow reporting an incorrect answer - C. Assert the answers with confidence to appear reliable - D. Hide cases where the AI did not find an answer
Response **B.** Trust is built by explaining **what the AI did** (what sources, what processing), not how it works internally (A). Assurance without proof (C) and concealment (D) destroy trust the first error discovered.
--- ### Question 9 — Build / Buy / Fine-tune A company wants to automatically classify its incoming emails into 6 business categories. Volume: 2,000/day. No particular sovereignty constraints. The recommended **default** choice is: - A. Build a model from scratch to control the entire chain - B. Immediately refine (fine-tuner) an open source model - C. Use an off-the-shelf API with a good prompt, and only consider fine-tuning if precision plateaus - D. Recruit a machine learning research team
Response **C.** “Buy” (off-the-shelf API) is the reasonable default for ~80% of cases: quick implementation, minimal maintenance, predictable cost. Fine-tuning is justified **later**, on proof: insufficient precision despite good prompting, volume which makes the economy favorable, training data available.A and D are disproportionate.
--- ### Question 10 — Responsible AI An application pre-sorting tool automatically rejects CVs (Curriculum Vitae) below a certain score, without human review. The **most serious** problem is: - A. The cost of API calls is too high - B. A decision with a strong impact on people is made without a human in the loop, with an uncontrolled risk of bias - C. The system response time is too long - D. The interface does not use the colors of the graphic charter
Response **B.** Recruitment is a high-risk use case (explicitly targeted by the European AI Act ⚠): high cost of error for people, potential bias (first name, address, etc.), obligation of transparency and recourse. The human in the loop cursor must be at least on “human approval” for rejections. A, C and D are side details.
--- ## 📊 Scale | Score | Interpretation | |---|---| | 9–10 | Excellent mastery — ready to put into practice on a real project | | 7–8 | Strong — review missed points before session 10 | | 5–6 | Fragile — reread slides 14–25 (value, metrics, cost-benefit, decision) | | <5 | Review the full session + redo exercises 1 and 2 with the interactive page |