Teacher guide — Q/K/V attention: from projections to causal output
Duration: 120 minutes
Positioning: Compute complete attention: W_Q, W_K, W_V, scores, √d_head, causal mask, softmax, value mixture, multi-head concatenation, and output projection.
Expected evidence: Established mechanisms; numerical simplifications are pedagogical.
Observable outcomes and preparation
- Distinguish query, key, and value.
- Run a complete numerical trace.
- Connect prefill, decode, and KV cache.
Before class, the instructor runs the worked case and lab, prints the exercise packet, prepares a four-column board—assumption, prediction, observation, delta—and checks that every mathematical notation is paired with dimensions. The demonstration must not become slide reading.
Diagnostic
- Explain in one sentence: Three learned projections. What observation would falsify your explanation?
- Explain in one sentence: Query-key compatibility. What observation would falsify your explanation?
- Explain in one sentence: Scaling. What observation would falsify your explanation?
Teaching decision: if two of three answers remain nominal or lack a validity condition, rebuild the vocabulary with a numerical example before any formula. A fluent but unfalsifiable answer does not count as mastery.
Timed plan
| Time | Activity | Observable evidence |
|---|---|---|
| 0–10 min | Individual diagnostic, then pair comparison | Three answers and one named uncertainty |
| 10–25 min | Situation and vocabulary | Annotated input → state → output diagram |
| 25–55 min | Develop the mechanism on the board | Shapes, assumptions, and intermediate calculation visible |
| 55–75 min | Worked case with deliberate errors | Reasoned correction, not only the right number |
| 75–95 min | Causal lab: predict, change one variable, run | Prediction / observation / delta table |
| 95–112 min | Exercises 1 and 2 with peer correction | Retained artifact and applied rubric |
| 112–120 min | Exit ticket and transfer | Mechanism, boundary, next experiment |
Teaching notes
1. Three learned projections
Each representation x produces q=xW_Q, k=xW_K, and v=xW_V. Query expresses what the position seeks; key describes how it can be found; value carries information to mix.
Working formula:
Q=XW_Q, K=XW_K, V=XW_V
2. Query-key compatibility
Dot product q·k measures alignment. In “Maya put down the book… She picked it up”, one head can learn that the query for “She” aligns with the key for “Maya”.
3. Scaling
As d_head grows, dot products can become large and saturate softmax. Dividing by √d_head keeps a more stable scale.
Working formula:
S=QKᵀ/√d_head
4. Causal mask and softmax
Before softmax, future positions receive −∞. Softmax turns each permitted row into positive weights summing to 1.
Working formula:
A=softmax(S+causal mask)
5. Weighted value mixture
Output is AV: weights choose how much of each value passes. Scores are not themselves the retrieved content.
Working formula:
O=AV
6. Multi-head, projection, and cache
Multiple heads compute different relations, their outputs are concatenated then projected by W_O. During decoding, past K/V are cached; the new query reads that cache without recomputing the prefix.
Working formula:
MHA(X)=Concat(head₁…head_h)W_O
Running the worked case
Reduced trace: q=[2,1], keys k_Maya=[1,1], k_book=[0,2]. The real head has d_head=4; our vectors show only 2 coordinates (a truncated illustration), but the scaling keeps the true dimension. Raw scores: 3 and 2; division by √4=2 gives 1.5 and 1. Softmax ≈ [0.62,0.38]. Output mixes 62% of v_Maya and 38% of v_book. For an earlier position, the causal mask would remove every future key.
Do not reveal the result at once. Ask learners to predict the next operation, its shape, and the expected sign. After each line ask: “What changed? What stayed fixed? Which assumption did we use?” A calculation error repaired with a causal chain is worth more than a guessed result.
Lab protocol
- Write a qualitative and, where possible, numerical prediction before touching a control.
- Change one variable only; retain a capture or record initial and final values.
- Explain the delta through the mechanism, not “the tool did that.”
- Test one boundary value and state where the model stops representing a real system.
Misconceptions
| # | Observable misconception | Grounded correction | Probe |
|---|---|---|---|
| 1 | “Three learned projections guarantees the outcome without assumptions or measurement.” | Each representation x produces q=xW_Q, k=xW_K, and v=xW_V. Query expresses what the position seeks; key describes how it can be found; value carries information to mix. | Ask for a counterexample, then restate the mechanism with its validity condition. |
| 2 | “Query-key compatibility guarantees the outcome without assumptions or measurement.” | Dot product q·k measures alignment. In “Maya put down the book… She picked it up”, one head can learn that the query for “She” aligns with the key for “Maya”. | Ask for a counterexample, then restate the mechanism with its validity condition. |
| 3 | “Scaling guarantees the outcome without assumptions or measurement.” | As d_head grows, dot products can become large and saturate softmax. Dividing by √d_head keeps a more stable scale. | Ask for a counterexample, then restate the mechanism with its validity condition. |
Boundary to maintain: Attention maps alone do not prove a causal explanation of the model’s overall behavior.
Probing questions
- If we remove or reverse Three learned projections, which output changes first, and what observation would show it?
- If we remove or reverse Query-key compatibility, which output changes first, and what observation would show it?
- If we remove or reverse Scaling, which output changes first, and what observation would show it?
- If we remove or reverse Causal mask and softmax, which output changes first, and what observation would show it?
- If we remove or reverse Weighted value mixture, which output changes first, and what observation would show it?
- If we remove or reverse Multi-head, projection, and cache, which output changes first, and what observation would show it?
Assessment
| Level | Criterion |
|---|---|
| 0 | Repeats terms without connecting input, transformation, and output. |
| 1 | Describes the chain but checks neither shape nor assumption. |
| 2 | Executes the case, explains the result, and names one limitation. |
| 3 | Transfers to a new case, compares an alternative, and proposes a measurement that could invalidate the choice. |
Exit threshold: level 2 on the worked case and at least one exercise; a memorized formula without interpretation remains level 1.
Observation and remediation protocol
During discussion, the instructor records evidence rather than impressions. Evidence of understanding contains a named object, a justified transformation, and a checkable consequence. If a learner gives the right result without a chain, ask for the preceding line. If the chain is coherent but the result is wrong, preserve the chain and isolate the arithmetic error. If vocabulary from another concept is used, compare both mechanisms in an input, state, output, cost, and boundary table. Remediation targets the first break only: vocabulary, shapes, operation, interpretation, or claim scope. After correction, use a neighboring case with a changed value; success on the same example does not prove transfer. For pair work, assign operator and verifier roles, then swap. The verifier does not supply the answer: they request an assumption, check the shape, and ask what observation could contradict the reasoning. The instructor retains the exit ticket and classifies the dominant break. The next session opens with a three-minute problem aimed at that break instead of repeating the whole lesson.
Differentiation
- Support: provide shapes and the first transformation; let the learner complete interpretation and boundary.
- Core path: worked case without result, lab with one assigned variable, diagnostic exercise.
- Extension: change one assumption, compare two mechanisms, and define the metric that would decide between them.
Post-session follow-up
Within twenty-four hours, return each annotated exit ticket with one priority, the exact resource to reopen, and a mini-case different from the worked case. Revision requires three items: a written prediction, a retained trace, and one sentence explaining the delta. At the next session, sample two submissions: one that repaired the break and one that remains ambiguous. Discuss them anonymously, then state the criterion that separates them. Do not use completion rate as evidence of mastery. Evidence of remediation is a correct chain on a new case with a stated boundary. If the same break appears in more than one third of the group, repair the support or demonstration before blaming learners.
Follow-up closes only when the new artifact shows the causal chain, check, and boundary—not merely when a file has been submitted.
Sources and evidence boundary
- Vaswani et al., “Attention Is All You Need”, NeurIPS (2017).
- Course source packet supplied by the owner; named-product details remain source-reported until primary verification.
Scope: Established mechanisms; numerical simplifications are pedagogical. These references support the session frame; they do not turn a reported product choice into an independently verified result.
Exit ticket
In no more than six lines: mechanism; calculation or trace; observation; boundary; evidence level; next experiment. The instructor marks one priority causal break for revision.