# Quiz with answers — Residual streams and attention over depth ### Question 1 Solve the worked-case variant: Increase the third checkpoint score from 2 to 2.4 in [0,1,2]. Recompute depth-softmax weights. - A. Depth scores [2,1,0] over {x₂₄,x₁₂,x₀} give softmax ≈ [0.67,0.24,0.09]. The mixture favors x₂₄, the most recent checkpoint, while the other two together retain 33%. - B. AttnRes is presented as a course mechanism and source-reported case study; exact Kimi K3 details require a primary source. - C. Under the assumption of roughly orthogonal contributions (‖F_l‖ = 0.5), the energy follows ‖x_l‖² ≈ 1 + 0.25·l: x₀’s share goes from 100% to 25% (l = 12), 14.3% (l = 24), 7.7% (l = 48). Diluted, never erased — addition is conservative. - D. Weights move from about [0.090,0.245,0.665] to [0.068,0.184,0.748]. Retrieval concentrates more on the third checkpoint while retaining nonzero contributions from the other two. **Answer: D.** Weights move from about [0.090,0.245,0.665] to [0.068,0.184,0.748]. Retrieval concentrates more on the third checkpoint while retaining nonzero contributions from the other two. The correct answer executes the requested change and gives a checkable result; the other texts do not close this calculation or trace. --- ### Question 2 Which causal order correctly connects the first three stages of “Residual streams and attention over depth”? - A. Residual connection → Dilution → Depth states - B. Depth states → Dilution → Residual connection - C. Dilution → Residual connection → Depth states - D. Residual connection → Depth states → Dilution **Answer: A.** Residual connection → Dilution → Depth states The chain follows the taught progression; reversing stages consumes a representation or state before it is produced. --- ### Question 3 If “Depth states” is removed, which diagnostic method is defensible? - A. Also change the data to amplify the difference. - B. Keep the same input, predict the first output that depends on “Depth states,” then compare the before/after trace. - C. Observe only the final output and invent the cause. - D. Conclude that the whole system fails before measuring. **Answer: B.** Keep the same input, predict the first output that depends on “Depth states,” then compare the before/after trace. One intervention and a prior prediction make the delta attributable to the removed mechanism. --- ### Question 4 Which verdict respects this session’s validity boundary? - A. The mechanism guarantees accuracy, speed, and stability for every workload. - B. One successful example proves the whole architecture is superior. - C. AttnRes is presented as a course mechanism and source-reported case study; exact Kimi K3 details require a primary source. - D. The mechanism name alone is enough for a production choice. **Answer: C.** AttnRes is presented as a course mechanism and source-reported case study; exact Kimi K3 details require a primary source. The correct answer bounds the conclusion; the others turn a local relation into a global guarantee. --- ### Question 5 Which evidence best matches the stated status of “Residual streams and attention over depth”? - A. The route loads without an error. - B. Every learner opened the file. - C. The same result is assumed on every hardware target. - D. Mixed: established mechanisms + source-reported Kimi K3-style choices. **Answer: D.** Mixed: established mechanisms + source-reported Kimi K3-style choices. Product or mechanism evidence must remain attributed and measured; availability and completion do not prove value. --- ### Question 6 When should a simpler baseline be preferred to “Blocking”? - A. Never: the newest mechanism wins by default. - B. When a controlled test shows equivalent quality with lower memory, latency, or complexity. - C. As soon as one memory metric falls, regardless of quality. - D. As soon as the diagram contains fewer components. **Answer: B.** When a controlled test shows equivalent quality with lower memory, latency, or complexity. The choice depends on a measured trade-off on the real workload, not novelty or one isolated metric. --- ### Question 7 A learner gets the right result but cannot explain “Dilution.” Which remediation is most useful? - A. Rebuild the first missing transformation, label its inputs and outputs, then test a neighboring case. - B. Accept the answer because the final number is correct. - C. Provide the final result a second time. - D. Change several variables and ask for an intuition. **Answer: A.** Rebuild the first missing transformation, label its inputs and outputs, then test a neighboring case. The remediation targets the first causal break and then requires transfer instead of rewarding a guessed result. --- ### Question 8 Which submission actually demonstrates the outcome “Evaluate storage cost.”? - A. A list of terms without causal relations. - B. A screenshot without values or interpretation. - C. A trace with starting data, transformations, observed result, boundary, and next experiment. - D. A confident claim without a baseline or threshold. **Answer: C.** A trace with starting data, transformations, observed result, boundary, and next experiment. The correct submission makes the reasoning reproducible and the verdict revisable by future measurement.