intermediate · Session 11

The complete pre-training pipeline

Connect raw data, filtering, tokenization, batches, targets, loss, updates, and the base model.

120 min6 mechanismscausal lab

What you will be able to do

Open the lab

Study method

Study this session as a causal investigation. Before every formula or interaction, write what you expect to change and what must remain fixed. During the calculation, retain units, shapes, and intermediate values so an error can be located without restarting at random. After the result, translate the number or state into one sentence about system behavior. Always finish with a counterexample or boundary value. This discipline separates understanding a mechanism from merely recognizing its vocabulary and makes the lab reproducible by another learner. Record the evidence that changed your initial prediction.

Build the mechanism step by step

1. Collect without accepting everything

The problem: A raw crawl contains the same viral article 3,000 times, API keys, and SEO spam. Trained as is, the model memorizes the duplicates — and can regurgitate them, secrets included. Fix it afterwards? The copies are already spread across every weight.

The idea: Filtering is a design stage at the head of the pipeline: fingerprint deduplication, secret detection, licensing and provenance rules. The article repeated 3,000 times must count once — before a single gradient is spent on it.

Why / at what price: Why so early: removing a document costs one comparison; unlearning it would cost a retraining run. The price: every filter has false positives — too aggressive, and it strips out the very diversity the gradient average was meant to capture.

Understanding check

Name the input, transformed state, output, and one required assumption. Then compare your chain with the explanation above.

2. Tokenize

The problem: The model does not consume letters: it needs discrete units from a finite inventory. Splitting by words blows up the vocabulary (“book”, “books”, “bookish”…); splitting by characters stretches every sentence and its cost. Where do you cut?

The idea: The tokenizer learns a subword vocabulary. “Maya reads a book” becomes 5 tokens for 4 words: [Maya][ reads][ a][ bo][ok] — “book” is split in two; boundaries do not follow words.

Why / at what price: Direct consequence: cost is counted in tokens, not words — here 5/4, i.e. +25%. The price: the vocabulary is frozen before training; a poorly covered domain (code, rare language, jargon) fragments into short tokens and pays more for the same information.

Understanding check

Name the input, transformed state, output, and one required assumption. Then compare your chain with the explanation above.

3. Build inputs and targets

The problem: You need millions of labeled examples, and nobody will hand-annotate the web. Where do targets come from? The text must supervise itself — without leakage: if a position can see its own answer, the loss drops to zero while learning nothing.

The idea: Shifting by one token manufactures supervision for free: for [A,B,C,D], input [A,B,C], target [B,C,D]. Each position predicts the next token without seeing the future; a sequence of L tokens yields L−1 training positions.

targets = tokens shifted left by one

Why / at what price: Free and unlimited — this is what makes pre-training possible at this scale. The price: supervision reduces to “the next token”. The model learns what the corpus makes follow, not what is true; target quality is exactly text quality.

Understanding check

Name the input, transformed state, output, and one required assumption. Then compare your chain with the explanation above.

4. Form batches

The problem: A cost-effective GPU processes thousands of positions in parallel, but sequences differ in length: 4 positions here, 9 there. How do you stack them into one rectangular tensor without corrupting the loss?

The idea: You align on the longest: T = 9; the short one gets 5 padded positions and a mask marks the emptiness. The mean loss divides by the 13 real positions, never by the 18 cells of the rectangle.

Why / at what price: The price of the rectangle: 5 cells out of 18 — about 28% of this batch’s compute — heat up nothing. And a forgotten mask corrupts the curve silently: the mean drops (sum ÷ 18) while not a single prediction improves.

Understanding check

Name the input, transformed state, output, and one required assumption. Then compare your chain with the explanation above.

5. Loss and update

The problem: At every position the model emits one score per vocabulary token — often 100,000 of them. The target is ONE token. You must turn those scores and that target into a single number that measures surprise, and punishes confident error hard.

The idea: Softmax turns scores into probabilities, then the loss takes −log p(target): p = 0.25 → 1.386; p = 0.50 → 0.693; p = 0.01 → 4.605. Gradients then redistribute that surprise across every parameter that contributed to it.

loss = −log p(target token)

Why / at what price: The −log slope blows up near zero, deliberately: being confident and wrong is very expensive. The price: the mean loss becomes a spoofable indicator — denominator, padding, or corpus mix can push it down with no real progress.

Understanding check

Name the input, transformed state, output, and one required assumption. Then compare your chain with the explanation above.

6. Base model and later stages

The problem: The loop stops: you have a base model that completes text. It does not follow instructions and can be toxic — and you must prove its capabilities when it may already have read your test questions during pre-training.

The idea: The pipeline ends in three branches: the base model; instruction tuning and alignment, which change behavior; and an evaluation set kept strictly separate from the corpus. That separation is decided at filtering time, not on test day.

Why / at what price: Why the seal is vital: one test question seen in pre-training turns the score into a memory measurement. The operating price: keeping the seal across terabytes takes permanent tooling — fingerprints, n-gram overlap — an engineering cost, not a formality.

Understanding check

Name the input, transformed state, output, and one required assumption. Then compare your chain with the explanation above.

Complete worked case

“Maya reads a book” becomes five tokens. The batch uses the first four as inputs and the last four as targets. If the model assigns 0.25 to the correct target, positional loss is −log(0.25) ≈ 1.386.

Reading method: write the data, state every object shape, perform one transformation, and interpret the result before continuing.

Validity boundary

A teaching pipeline omits distributed storage, security, data policies, and many production quality controls.

Evidence status: Established mechanisms; numerical simplifications are pedagogical.

Quick checks

1. What does Collect without accepting everything?

Filtering is a design stage at the head of the pipeline: fingerprint deduplication, secret detection, licensing and provenance rules. The article repeated 3,000 times must count once — before a single gradient is spent on it.

2. What does Tokenize?

The tokenizer learns a subword vocabulary. “Maya reads a book” becomes 5 tokens for 4 words: [Maya][ reads][ a][ bo][ok] — “book” is split in two; boundaries do not follow words.

3. What does Build inputs and targets?

Shifting by one token manufactures supervision for free: for [A,B,C,D], input [A,B,C], target [B,C,D]. Each position predicts the next token without seeing the future; a sequence of L tokens yields L−1 training positions.

4. What does Form batches?

You align on the longest: T = 9; the short one gets 5 padded positions and a mask marks the emptiness. The mean loss divides by the 13 real positions, never by the 18 cells of the rectangle.

Sources and evidence boundary

Scope: Established mechanisms; numerical simplifications are pedagogical.