Quiz — Session 4: RAG, giving memory to the model

Program: Applied AI — Yann Isola · Intermediate level 11 questions · one correct answer per question · explained answer key at the end

Reminder: RAG = Retrieval-Augmented Generation: retrieve relevant evidence before generating an answer.

Question 1 — Which of the following statements about knowledge of an LLM (Large Language Model) used without RAG is FALSE?

A Its knowledge ends at the training cut-off date. B It has never seen your company’s internal documents. C It stores knowledge in compressed, lossy model weights. D It can recite verbatim any public document from its training data.

Question 2 — The central analogy of the RAG seen in class is:

A Give glasses to a myopic model. B Transform a closed-memory exam into an open-book exam. C Replace the brain of the model with a database. D Teach the model a new language.

Question 3 — What is the correct sequence of the ingest pipeline (offline)?

A Document embedding → chunking → indexing. B Cutting into chunks → embedding of each chunk → indexing of the vectors. C Indexing of documents → division → generation. D Question embedding → retrieval → generation.

Question 4 — In the query pipeline (online), why do we vectorize the question with the same embedding model as that used for the chunks?

A To save memory. B Because it’s faster. C So that question and chunks live in the same geometric space, condition for proximity to measure similarity of meaning. D Because embedding models only work on questions.

Question 5 — A chunk that is much too large (for example 5,000 tokens) mainly poses what problem?

A It mixes several subjects: its embedding becomes a fuzzy “average”, less discriminating for retrieval. B It is illegal in most vector bases. C It makes the embedding model slower to train. D None: the larger the chunk, the better the retrieval.

Question 6 — Why do we add an overlap between consecutive chunks?

A To artificially increase the size of the index. B To prevent information straddling a cutting boundary from being lost or cut in two. C So that the model generates longer answers. D To compress documents.

Question 7 — When cutting, the rule regarding tables is:

A Cut them into individual lines for smaller chunks. B Delete them, because the models do not read the tables. C Keep them whole in a single chunk: a line separated from its header becomes unintelligible. D Systematically convert them into images.

Question 8 — A user searches for “error REF-2024-8812” and the purely vector RAG finds nothing relevant. What is the most likely explanation and solution?

A The generation model is too small; you need a larger model. B Exact codes have little semantic content: their embedding does not distinguish them well. Solution: hybrid search, combining vectors and keyword search. C The context window is saturated; it must be emptied. D The code is too new for embedding; the model must be retrained.

Question 9 — Your RAG answers next to the question. You inspect and see that the retrieved chunks are off-topic. The correct diagnosis is:

A The generation model hallucinates; it must be replaced. B Retrieval failed: the model worked correctly with bad ingredients. The division, index or search must be corrected. C The user asked the wrong question; nothing to correct. D It’s a random bug, rerunning the query will be enough.

Question 10 — What is the role of the instruction “Answer only from the context provided; if the answer is not there, say so”?

A Accelerate generation. B Reduce the cost of queries. C Prevent the model from filling in the gaps with its internal memory, and allow honest refusal rather than plausible invention. D Force the model to answer even when evidence is absent.

Question 11 — A self-correction step asks the same model whether its answer is grounded. Why is that insufficient to open the evidence gate?

A A model can never reread its own answer. B The same error can be repeated; require verifiable citations and thresholds calibrated on an external evaluation set. C Only HyDE can verify an answer. D Reranking replaces evaluation entirely.