# Quiz — Advanced Level, Session 8 # “Context, reliability & provenance” **Program:** Applied AI — Yann Isola **Format:** 10 MCQs (MCQ = Multiple Choice Questionnaire), certification level *Claude Certified Architect* **Duration:** 20 minutes — only one correct answer per question unless otherwise stated **Threshold:** 7/10 > ⚠ The figures cited (context windows, 50% batch reduction, 100,000 request limit, 24-hour SLA) reflect the documentation at the time of writing and are **volatile**: check the official Anthropic documentation. SLA = Service Level Agreement. --- ### Q1 — Nature of the API and memory A developer is surprised: “I sent the client file to the model three requests ago, why doesn’t it remember it anymore? » What is the right architect's explanation? - A. The model has a 30 minute memory; beyond that, the data must be returned - B. The API is stateless: the model only sees what is returned in the current request; history management is entirely the responsibility of the application - C. You must activate the `memory: true` option in the query parameters - D. Prompt caching would have preserved the file; it is its absence that causes forgetting **Answer: B.** The Messages API is stateless: no server-side history. A and C invent non-existent mechanisms. D confuses cache and memory: the cache reduces the **cost** of reprocessing an identical prefix returned by the client — it never eliminates the need to resend it. --- ### Q2 — Saturation arithmetic Window of 200,000 tokens ⚠. Fixed costs (system + tool definitions): 7,000 tokens. Exit reserve: 8,000 tokens. Each full spin (trade + raw tool result) adds ~3,300 tokens to the history. Order of magnitude of the saturation turn? - A. ~25 turns - B. ~56 turns - C. ~150 turns - D. No saturation: old tours are automatically eliminated by the API **Answer: B.** (200,000 − 7,000 − 8,000) / 3,300 ≈ 56. D is the central pitfall: the API does not eliminate anything automatically — if the context exceeds the window, the query fails; any truncation/compression strategy is applicable. Architect's reflex: this calculation is done **before** writing the agent. --- ### Q3 — Choice of context strategy A long-term investigation agent must both remember decisions made 100 turns ago and reason carefully about the last 5 exchanges (exact tool results included). What strategy? - A. Strict sliding window on the last 20 revolutions: simple and without compression cost - B. Full summary of all history at each turn: maximum compactness - C. Hybrid: cumulative summary of old turns + last N turns kept verbatim - D. Increase `max_tokens` to enlarge the context window **Answer: C.** The hybrid preserves the overview (summary) and the recent detail (verbatim). A loses old decisions (amnesia). B destroys the accuracy of recent rounds and costs a compression call each round. D is a vocabulary trap: `max_tokens` caps the **output**, it does not enlarge the context window. --- ### Q4 — Reliability of the context summary In a digest-compression architecture, which statement is correct? - A. The summary is lossless if we use the same model as the agent -B.The summary is a model output: lossy compression, possible hallucination; in a regulated context, the full history must be kept out of context (audit log) - C. At temperature 0, the summary is guaranteed to be faithful to the history - D. The summary must include all raw tool results to remain usable **Answer: B.** A generated summary is intrinsically lossy and fallible — it can be controlled (conservation instructions: identifiers, decisions, open leads) but cannot be certified. A and C confuse inference parameters and fidelity guarantees. D is the opposite of the goal: reinjecting the crude cancels the compression. --- ### Q5 — PostToolUse hook and context pollution A `search_transactions` tool returns 40 fields per transaction and up to 10,000 rows. The agent uses 4 fields and a few reported lines. What is the best practice? - A. Insert everything in context: the more information the model has, the better it reasons - B. A PostToolUse hook which logs the full raw result (audit), then inserts only the relevant fields into the context with a row ceiling - C. Delete the raw result and only log the filtered version, to save storage - D. Ask the model, in the prompt, to ignore irrelevant fields **Answer: B.** Double gesture: raw → newspaper (origin), filtered + capped → context (hygiene). A is the definition of context pollution — irrelevant information **degrades** performance (less is more) and re-invoices each turn. C destroys the chain of provenance: the listener must be able to compare raw and ctx. D lets noise consume tokens and attention. --- ### Q6 — Scratchpad pattern What is the decisive advantage of the “investigation-scratchpad.md” pattern (persistent external memory) compared to the summary in context alone? - A. It makes the summary in context useless in all cases - B. It guarantees the accuracy of the noted facts, because the file is not generated by the model - C. It provides a selective memory that survives compressions, restarts and resets, inspectable by a human and transferable to another agent - D. It is automatically synchronized with the context window by the API **Answer: C.** Persistence out of context + inspectability + transmissibility: this is what the summary (ephemeral, regenerated, in context) does not give. A is false: the two complement each other (different roles). B is false: the scratchpad is written **by** the model — hence the requirement to attach the sources of the “established facts”. D invents a mechanism: the API does not know this file; proofreading and updating are application-based (tools). --- ### Q7 — Provenance verification A generated report cites `{"tool": "check_sanctions", "call_id": "call_0042", "record_id": "SANC-2211-08"}`. What should the application do before considering the assertion as sourced? - A. Nothing: a structured quote produced at temperature 0 is reliable - B. Check that the format of `record_id` respects the expected regular expression - C. Resolve pointer against audit log: `call_id` exists, and `record_id` is in the raw result of this call - D. Ask the model again to confirm their quote in a second call **Answer: C.** A model can hallucinate a plausible citation (good format, bad content): the citation is a **pointer to resolve**, not a proof.A and B are fooled by surface plausibility. D has the forger check it out himself — a second call can confidently confirm a made-up quote. A provenance chain without mechanical verification of pointers is compliance theater. --- ### Q8 — Reproducibility For a regulatory report, management requests the guarantee of being able to “regenerate exactly the same report” from the same inputs. Correct response from the architect? - A. Yes: temperature 0 + pinned model version + seed ⚠ guarantee identical bit-to-bit outputs - B. No, and this is why generative AI is unusable in the regulated sector - C. Regeneration is quasi-deterministic but not guaranteed bit by bit (inference infrastructure); the guarantee of conformity is based on the exhaustive audit log of the output actually produced and all its context - D. Yes, provided you use the Batches API, which is deterministic by design **Answer: C.** “Deterministic-ish”: setting temperature, version and seed reduces the variance but the inference infrastructure (parallelism, server batching) introduces residual variations. Compliance is based on what was **logged**, not what could be regenerated. A over-promises (classic contractual error), B over-reacts, D invents a property of the Batches API. --- ### Q9 — API Batches: counting Your batch of 80,000 requests displays `processing_status: "ended"`. Which statement is correct? - A. All requests were successful; results are in order of submission - B. Each query has an individual status (`succeeded`, `errored`, `canceled`, `expired`) and the order of results is not guaranteed: counting is done query by query, correlated by `custom_id` - C. Error requests were automatically resubmitted by the API - D. The results are pushed by webhook; no counting is necessary **Answer: B.** `ended` means that batch processing is complete, not that everything was successful. The `custom_id`, provided by you at submission, is the only reliable correlation key (non-guaranteed order) — and it also serves the recovery idempotency and audit chain. C and D invent automation: retry and counting are your responsibility. --- ### Q10 — Economical batch + cache optimization Non-interactive nightly reprocessing of 80,000 files sharing a system prompt of 3,200 tokens. Which configuration minimizes the cost, and why? - A. Synchronous calls with cache: the cache is incompatible with batches - B. Batches API only: the 50% reduction ⚠ makes the cache irrelevant - C. API Batches with the system block in `cache_control`: the batch reduction of 50% ⚠ (input and output) combines with the reduced rate of cache reads on the shared prefix - D. Parallelized synchronous calls: batch is more expensive because it includes the 24-hour SLA ⚠ **Answer: C.** The two levers are cumulative: −50% batch on the whole, plus the cache discount on the common prefix — this is the “batch + cache” pattern of maximum optimization for the non-interactive mass. A is false (the cache can be used in batches ⚠ — check the current doc for the details). B leaves money on the table: out of 80,000 requests, the shared prefix weighs heavily. D reverses reality: the batch is cheaper precisely because it gives up real time. ---## Quick correction grid | Q | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |---|---|---|---|---|---|---|---|---|---|----| | R | B | B | C | B | B | C | C | C | B | C | **Topics covered:** statelessness (Q1), context budget (Q2), compression strategies (Q3, Q4), hygiene and hooks (Q5), external memory (Q6), provenance and verification (Q7), reproducibility (Q8), Batches API (Q9), batch optimization + cache (Q10).