❓ Quiz — intermediate/session-01

Program : Applied AI — Intermediate Level Instructor: Yann Isola Format: 12 multiple choice questions — only one correct answer per question. Recommended duration: 18 minutes independently, or 4–5 questions selected by show of hands at the end of the session. Recommended threshold for success: 8/12.

Question 1 — (Memorization) The innovation of 2017

Which 2017 publication introduced “transform” architecture?

HAS ImageNet Classification with Deep Convolutional Neural Networks B Attention Is All You Need C Playing Atari with Deep Reinforcement Learning D Language Models are Few-Shot Learners

Question 2 — (Understanding) What the model actually “reads”

When you send the phrase “Hello, how are you?” » to a language model, what does it actually process?

HAS Each letter, one by one B Each whole word, like in a dictionary C Tokens: fragments of text of around 3–4 characters on average ⚠ D An image of the sentence, analyzed by optical recognition

Question 3 — (Application) The riddle of counting letters

A model gets it wrong when counting the number of “r”s in “strawberry”. What is the MOST accurate explanation?

HAS The model never encountered this word during training B The word is divided into opaque tokens: the model has no direct access to individual characters C The model does not understand English D The model's memory is saturated with conversation

Question 4 — (Understanding) Embeddings

What does the famous equation “king − man + woman ≈ queen” mean?

HAS The model stores a database of royal family relationships B Words are vectors, and meaning relationships correspond to coherent directions in vector space C The model applies a hand-coded grammar rule for gender D It's a meaningless coincidence, often misquoted

Question 5 — (Understanding) The role of attention

What is the purpose of the attention mechanism in a transformer?

HAS To filter inappropriate content before responding B To allow each token to dynamically evaluate the relevance of all other tokens in the context C To permanently memorize user conversations D Compress text to save memory

Question 6 — (Application) The Pipeline Analogy

Your company wants an existing model to learn the exact format of your internal audit reports. In the course analogy, this step corresponds to:

HAS University (pre-training) B Finishing school (post-training / RLHF) C Corporate onboarding (fine-tuning) D The final exam (evaluation / benchmark)

Question 7 — (Memorization) The pre-training task

What is the unique task that pre-trains a large language model?

HAS Correctly answer questions asked by humans B Translate texts between multiple languages C Predict the next token in a text D Classify texts into categories (spam, sentiment, subject, etc.)

Question 8 — (Application) The inference bottleneck

A chatbot in production responds too slowly. The service provider offers to double the raw computing power of the servers. Why is this proposal likely ineffective?

HAS Because the inference speed is mainly limited by the memory bandwidth: each generated token requires passing all the model parameters from memory B Because the models have a fixed speed defined by their user license C Because doubling the calculation also doubles the response time D Because the slowness always comes from the internet network

Question 9 — (Understanding) Scaling laws

What do “scaling laws” describe?

HAS European regulations governing the maximum size of AI models B The regular and predictable improvement in performance when we increase data, calculation and parameters together C Year-over-year growth in AI vendor pricing D How quickly a model forgets old information

Question 10 — (Application/synthesis) The double contribution of transformers

Before 2017, Recurrent Neural Networks (RNNs) processed text word by word. What TWO problems did the transformers solve simultaneously?

HAS The cost of electricity and the size of data centers B Machine translation and voice recognition C The forgetting of distant dependencies in the text (resolved by attention) and the impossibility of parallelizing the training (resolved by a parallel architecture exploiting GPUs) D Hallucinations and model biases

Question 11 — (Understanding) The raw material of LLMs

In the FineWeb type data pipeline presented in the session (the one that Andrej Karpathy uses in his reference video), what is the correct order of steps to go from the raw web to the pre-training dataset?

HAS Tokenization → crawl → filtering → training B Crawl (CommonCrawl) → extracting text from HTML → filtering (language, quality) → deduplication C Deduplication → crawl → human annotation of each page → extraction D Filtering → tokenization → crawl → deduplication

Question 12 — (Analysis) The deduplication paradox

The FineWeb authors compared two strategies: deduplicating all 96 crawls globally, or deduplicating each crawl independently. What did they notice?

HAS Global deduplication results in better models because it removes more duplicates B Both strategies give identical results, deduplication is useless C Crawl-by-craw deduplication gives better models: the global version over-represented old, poor-quality data D Global deduplication is better but too expensive, so they chose the crawl version to save money

Quiz — Session 1: Fundamentals: Transformers & Tokenization