Final quiz — Session 10: complete program evaluation

Program : Applied AI — Intermediate Level — Instructor: Yann Isola Format: 12 MCQ (multiple choice questionnaire) — only one correct answer per question — recommended duration: 12 minutes. Special feature: this final quiz covers all 10 sessions of the program. One question per session, in the order of the course.

Question 1 — [Session 1 — Foundations] Why might an LLM (Large Language Model) fail to count the letters of the word “unconstitutionally”?

HAS Because the model was not trained on French words B Because the model does not see letters but tokens (text division units): the word is divided into a few fragments, and the individual letters do not exist in its representation C Because the context window is too short for such a long word D Because the temperature is set too high by default

Question 2 — [Session 2 — Prompting] Among these four prompts, which one best applies good professional practices?

HAS “Write an email. » B “Write a professional email, please, it’s important, do your best, thank you very much. » C “You are a sales assistant. Write a follow-up email (≤ 120 words) for a quote sent 10 days ago to customer {name}. Tone: cordial, without pressure. End with an open question. Here are two examples of the expected style: {example_1} {example_2} » D “Write the best follow-up email ever written, be creative and don’t hesitate to improvise on the context. »

Question 3 — [Session 3 — Structured output & temperature] You build a pipeline that extracts invoice amounts into automatically processed JSON (JavaScript Object Notation, structured data format). Which setting is most suitable?

HAS High temperature (≈ 1.0) for the model to find creative solutions to ambiguous invoices B Low temperature (≈ 0) + structured output scheme imposed, because we want reproducibility and a guaranteed machine-usable format C Average temperature (≈ 0.7) which is the best universal compromise D Temperature has no effect on output format stability

Question 4 — [Session 4 — RAG] What is the principle of RAG (Retrieval-Augmented Generation)?

HAS Retrain the model on company documents so that it remembers them B Search, at the time of the question, the relevant passages in a documentary database, then inject into prompt for the model to respond from these sources C Increase context window size by paying vendor option D Generate synthetic documents to enrich the knowledge base

Question 5 — [Session 5 — Tool calling] When a model “calls a tool”, who actually executes the action (database query, sending an e-mail, etc.)?

HAS The model itself, directly on your servers B The model provider, in its cloud C Your application code : the model only issues a structured request, and you execute (or deny) the call, with your permissions and logging D A trusted third-party service certified by the supplier

Question 6 — [Session 6 — Agentic loop] What fundamentally distinguishes an agent from a simple model call?

HAS An agent uses a larger model B An agent operates in loop : he observes the result of his actions, decides what to do next, and starts again until he reaches the objective or a stopping condition - hence the importance of limits (iteration budget, human in the loop) C An agent does not need a system prompt D An agent cannot hallucinate because he checks his answers

Question 7 — [Session 7 — Multi-agents & MCP] What is Model Context Protocol (MCP) used for?

HAS To make the agents communicate with each other by token telepathy B To standardize the way a model connects to tools and data sources: an MCP server exposes its capabilities once, and any compatible client can use them without custom integration C To compress the context to reduce costs D To encrypt the exchanges between the model and the user

Question 8 — [Session 8 — Production] Your AI application receives HTTP 429 errors from the provider. What is happening and what are you doing?

HAS The provider's server is down; you must change supplier immediately B Your API (Application Programming Interface) key is invalid; it must be regenerated C You exceed the rate limit (rate limit); you have to slow down and try again with an exponential backoff (waiting which doubles with each attempt), and ultimately smooth or distribute the load D The prompt is too long; it must be truncated

Question 9 — [Session 9 — Evaluation] Why is “it looks good on my 5 tests” not an acceptable evaluation before going into production?

HAS Because a minimum of 5,000 tests are required for any evaluation B Because 5 tests chosen by the developer measure neither variability, nor edge cases, nor performance by segment : you need a representative evaluation game, defined metrics and an explicit decision threshold C Because only external certification from the supplier is authentic D Because manual evaluation is prohibited by the GDPR (General Data Protection Regulation)

Question 10 — [Session 10 — Governance] Your company deploys an AI candidate pre-screening assistant. According to the AI Act (European regulation on artificial intelligence) ⚠, which category does this system fall into, and with what main consequence?

HAS Minimal risk: no obligation, recruitment is for free internal use B Limited risk: it is enough to indicate to candidates that an AI is used C High risk : recruitment is included in sensitive areas — reinforced obligations (risk management, data quality, documentation, human supervision) D Unacceptable risk: any use of AI in recruitment is prohibited in the EU (European Union)

Question 11 — [Institutional sovereignty] Why is model liquidity a governance objective?

HAS Because it allows you to change model or supplier without rewriting the application, if prices, rules or performance change B Because it guarantees that all models give exactly the same answer C Because it removes the need for evaluations D Because it requires only using open-weight models

Question 12 — [Institutional Sovereignty] What asset becomes the company's true defensible advantage in a mature AI system?

HAS The commercial name of the model used B The total number of tokens consumed each month C Ontology and business context: objects, permissions, workflows, decisions, human evaluations and corrections D Never logging interactions to protect privacy