❓ Quiz — advanced/session-07

Q1 — When the Chain of Thought harms

One team adds “Think step by step” to a prompt that verbatim copy an IBAN number (IBAN = International Bank Account Number) from a document. Accuracy drops. What is the most likely explanation?

HAS CoT increases the effective temperature of the model B The model “over-reasons” and paraphrases instead of copying; the CoT is counterproductive on verbatim extraction C The CoT has no possible effect on accuracy, the problem lies elsewhere D CoT exceeds budget max_tokens , systematically truncating the IBAN

Q2 — Design of a few-shot game

You have a budget of 6 few-shot examples for a classification. What composition maximizes reliability?

HAS 6 examples of the most common case in production, to stick to real distribution B 6 examples each covering a different class, all perfectly nominal C 1-2 nominal cases, 2-3 ambiguous borderline cases, 1 out-of-scope/refusal case, varying the surface attributes D The maximum number of examples possible: reliability increases linearly with their number

Q3 — Prefilling: exact API behavior

You send {"role": "assistant", "content": "<resultat>"} in last message with stop_sequences: ["</resultat>"]. The model generates 42</resultat>…. What does it contain response.content and what is worth stop_reason ?

HAS <resultat>42</resultat> — stop_reason: "end_turn" B 42 — stop_reason: "stop_sequence" C <resultat>42 — stop_reason: "stop_sequence" D 42</resultat> — stop_reason: "stop_sequence"

Q4 — Prefill trap

Which of these prefills causes a API error ?

HAS {"role": "assistant", "content": "{"} B {"role": "assistant", "content": "Voici l'analyse : "} C {"role": "assistant", "content": "[ANALYSTE]"} D {"role": "assistant", "content": "<reponse>"}

Q5 — Long context: placement of instructions

Prompt of 80,000 tokens: three contracts attached and a critical analysis instruction. What placement maximizes compliance with the instructions?

HAS Directions at beginning only — model reads in order B Instructions in the middle, between the documents, as close as possible to the data C Documents tagged in XML at the beginning; critical instruction stated at the beginning And repeated at the end D It doesn't matter: attention is uniform across the entire context window

Q6 — Examples vs instructions

A system prompt requires "reply in JSON only", but two of the five few-shot examples contain an explanation sentence before the JSON. In production, the model often adds a phrase before its JSON. What diagnosis?

HAS Model bug: system instructions always take precedence over examples B Temperature too high: lowering it to 0 will be enough C Examples/instructions conflict: the model imitates the examples, which often override the opposite instruction — correct the examples D You must add “VERY IMPORTANT” in front of the JSON instruction

Q7 — Canonical production pattern

In the boss classification → routage → génération spécialisée → validation , which statement is correct?

HAS Routing should be handled by a dedicated LLM call for flexibility B The routing is deterministic code (switch on the structured output of the classification); you don't pay a model for one if C Validation should always be a judge LLM, never programmatic rules D Classification and generation must use the same model to ensure consistency

Q8 — Negative prompting

Why is “If information is missing, use the value “unknown”” more reliable than “Never invent information” alone?

HAS Models do not understand French grammatical negation B The negative statement describes a void without defining the replacement behavior; the positive instruction specifies exactly what to do in the problem case C Negative instructions are ignored by the system prompt, they only work as a user message D No measurable difference: it’s a stylistic preference

Q9 — Temperature and Chain of Thought

For an extraction pipeline with structured CoT, what temperature setting and why? And in what case would we voluntarily do the opposite?

HAS High temperature still: the CoT needs creativity to explore B Low temperature (often 0) for reproducible reasoning paths; the opposite — sampling several CoTs at higher temperatures then majority vote (self-consistency) — is reserved for critical decisions because they are costly C Temperature has no effect when a CoT is present in the prompt D Temperature 0 makes the CoT useless since the model becomes deterministic

Q10 — The “contract” prompt system

A prompt production system for a ticket sorting agent should, at a minimum, contain four clauses. Which ones?

HAS Detailed persona, company history, sample tickets, signature B Explicit rules (scope, sources), exact output format, refusal policy with imposed wording, triggers for escalation towards humans C Temperature, max_tokens, stop_sequences, model name D Positive instructions only — any mention of refusal degrades performance

Quiz — Advanced Level, Session 7

“Advanced prompt engineering”