# Final quiz — Session 10: complete program evaluation **Program:** Applied AI — Intermediate Level — Instructor: Yann Isola **Format:** 12 MCQs (multiple choice questionnaire) — only one correct answer per question — recommended duration: 12 minutes. **Special feature:** this final quiz covers **all 10 sessions** of the program. One question per session, in the order of the course. --- **Q1. [Session 1 — Foundations] Why can an LLM (Large Language Model) fail to count the letters of the word “unconstitutionally”?** - A. Because the model was not trained on French words - B. Because the model does not see letters but **tokens** (text division units): the word is divided into a few fragments, and the individual letters do not exist in its representation - C. Because the context window is too short for such a long word - D. Because the temperature is set too high by default **Q2. [Session 2 — Prompting] Among these four prompts, which one best applies good professional practices?** - A. “Write an email. » - B. “Write a professional email, please, it’s important, do your best, thank you very much.” » - C. “You are a sales assistant. Write a follow-up email (≤ 120 words) for a quote sent 10 days ago to customer {name}. Tone: cordial, without pressure. End with an open question. Here are two examples of the expected style: {example_1} {example_2} » - D. “Write the best follow-up email ever written, be creative and don’t hesitate to improvise on the context. » **Q3. [Session 3 — Structured output & temperature] You build a pipeline that extracts invoice amounts into automatically processed JSON (JavaScript Object Notation, structured data format). Which setting is most suitable?** - A. High temperature (≈ 1.0) for the model to find creative solutions to ambiguous invoices - B. Low temperature (≈ 0) + structured output scheme imposed, because we want **reproducibility** and a format guaranteed to be used by the machine - C. Average temperature (≈ 0.7) which is the best universal compromise - D. Temperature has no effect on the stability of the output format **Q4. [Session 4 — RAG] What is the principle of RAG (Retrieval-Augmented Generation)?** - A. Retrain the model on company documents so that it memorizes them - B. Search, at the time of the question, the relevant passages in a documentary database, then **inject them into the prompt** so that the model responds from these sources - C. Increase the size of the context window by paying a provider option - D. Generate synthetic documents to enrich the knowledge base **Q5. [Session 5 — Tool calling] When a model “calls a tool”, which actually executes the action (database query, sending an e-mail, etc.)?** - A. The model itself, directly on your servers - B. The model provider, in its cloud - C. **Your application code**: the model only issues a structured request, and it is you who execute (or refuse) the call, with your authorizations and your logging - D. A trusted third-party service certified by the supplier **Q6. [Session 6 — Agentic loop] What fundamentally distinguishes an agent from a simple model call?** -A.An agent uses a larger model - B. An agent operates in a **loop**: it observes the result of its actions, decides what to do next, and starts again until it reaches the objective or a stopping condition — hence the importance of limits (iteration budget, human in the loop) - C. An agent does not need a system prompt - D. An agent cannot hallucinate because he checks his answers **Q7. [Session 7 — Multi-agents & MCP] What is MCP (Model Context Protocol) used for?** - A. To make the agents communicate with each other by token telepathy - B. To standardize the way a model connects to tools and data sources: an MCP server exposes its capabilities once, and any compatible client can use them without custom integration - C. To compress the context to reduce costs - D. To encrypt the exchanges between the model and the user **Q8. [Session 8 — Production] Your AI application receives HTTP 429 errors from the provider. What is happening and what are you doing?** - A. The supplier's server is down; you must change supplier immediately - B. Your API (Application Programming Interface) key is invalid; it must be regenerated - C. You exceed the **rate limit**; you have to slow down and try again with an exponential backoff (waiting which doubles with each attempt), and ultimately smooth or distribute the load - D. The prompt is too long; it must be truncated **Q9. [Session 9 — Evaluation] Why is “it looks good on my 5 tries” not an acceptable evaluation before going live?** - A. Because a minimum of 5,000 tests are required for any evaluation - B. Because 5 tests chosen by the developer measure neither variability, nor borderline cases, nor performance **by segment**: a representative evaluation set, defined metrics and an explicit decision threshold are required - C. Because only external certification from the supplier is authentic - D. Because manual evaluation is prohibited by the GDPR (General Data Protection Regulation) **Q10. [Session 10 — Governance] Your company deploys an AI candidate pre-screening assistant. According to the AI Act (European regulation on artificial intelligence) ⚠, which category does this system fall into, and with what main consequence?** - A. Minimal risk: no obligation, recruitment is for free internal use - B. Limited risk: it is enough to indicate to candidates that an AI is used - C. **High risk**: recruitment is in sensitive areas — reinforced obligations (risk management, data quality, documentation, human supervision) - D. Unacceptable risk: any use of AI in recruitment is prohibited in the EU (European Union) **Q11. [Institutional sovereignty] Why is model liquidity a governance objective?** - A. Because it allows you to change model or supplier without rewriting the application, if prices, rules or performance change - B. Because it guarantees that all models give exactly the same answer - C. Because it eliminates the need for evaluations - D. Because it requires using only open-weight models **Q12. [Institutional Sovereignty] What asset becomes the company's true defensible advantage in a mature AI system?** -A.The commercial name of the model used - B. The total number of tokens consumed each month - C. Ontology and business context: objects, permissions, workflows, decisions, human evaluations and corrections - D. Never logging interactions to protect privacy --- ## Corrected & explanations **Q1 → B.** The model manipulates tokens, not characters. “Unconstitutionally” becomes 3–6 tokens ⚠ depending on the tokenizer: asking to count your letters is like asking to count the bricks of a wall while only seeing whole sections. (Session 1 — this was the very first “surprise” of the course.) **Q2 → C.** Role + specific task + constraints (length, tone) + variables + examples (few-shot, learning by examples): the complete recipe. A is empty, B confuses politeness and context, D maximizes unpredictability where we want control. (Session 2.) **Q3 → B.** Automated pipeline = reproducibility and guaranteed format. “Creativity” (high temperature) is a fault here: an ambiguous invoice must be **reported**, not imaginatively interpreted. (Session 3.) **Q4 → B.** The RAG does not modify the model (unlike A): it enriches the prompt at the time of the request with the retrieved passages. This is why it is continuously updatable and traceable (sources can be cited). (Session 4.) **Q5 → C.** The model **requests**, your code **has**. This is the foundation of tool security: validation, permissions and logging remain on your side. (Session 5 — and this is also the basis of “responsibility” seen in Session 10.) **Q6 → B.** The observation → decision → action loop is the definition of the agent. Corollary: without limits (iterations, budget, human validation for irreversible actions), a loop can get out of control. (Session 6.) **Q7 → B.** MCP standardizes the tool/data connection: “a universal port” instead of tailor-made integration by client-tool pair. (Session 7.) **Q8 → C.** 429 = “Too Many Requests”, rate limit signal, no failure. Engineer response: retry with exponential backoff + load smoothing. Confusing 429 with server failure leads to poor architectural decisions. (Session 8.) **Q9 → B.** An evaluation worthy of the name: representative case set, explicit metrics, go/no-go threshold, and — final lesson of Session 10 — segmentation by group to uncover the biases that the average hides. (Session 9.) **Q10 → C.** Recruitment is explicitly in the **high risk** areas of the AI Act ⚠: authorized, but subject to heavy obligations, including human supervision. Neither forbidden (D), nor harmless (A, B). The amounts of sanctions and the application timetable are changing ⚠: check the state of the law at the time of the project. (Session 10.) **Q11 → A.** Model liquidity is a strategic insurance: it transforms the supplier into a replaceable component. It requires stable input/output contracts, comparable evaluations and a model-agnostic application layer. **Q12 → C.** Models are becoming commonplace; the business context accumulates. The ontology, permissions, workflows, audit trails and human corrections form the flywheel that the company must have. --- ### Indicative scale- **11–12:** complete mastery of the course — the final project should confirm this. - **8–10:** solid; review the answers to missed questions, these are probably sessions to be re-read diagonally. - **6–7:** the fundamentals are there; repeat the guides for the sessions concerned before applying in a professional context. - **< 6:** re-browse the media; offer an individual interview with the trainer to target revisions.