# Quiz with answers — Pre-training systems and optimization ### Question 1 Solve the worked-case variant: Increase the target logit from 2 to 2.4 while keeping the other logits at 1 and 0. Recompute softmax and cross-entropy for target 0. - A. For logits [2,1,0] and target 0, softmax ≈ [0.665,0.245,0.090], so CE ≈ 0.408. The lab changes learning rate, gradient, and weight, then shows fields required for recovery. - B. Distributed performance depends on hardware, network, model size, and implementation; no lab estimate is a benchmark. - C. Softmax moves from about [0.665,0.245,0.090] to [0.748,0.184,0.068]. Loss −log p(target) falls from about 0.408 to 0.291: raising the target logit relative to the others reduces loss. - D. Stable log-softmax subtracts log-sum-exp. Loss then selects the target log-probability. Larger logits matter only relative to others. **Answer: C.** Softmax moves from about [0.665,0.245,0.090] to [0.748,0.184,0.068]. Loss −log p(target) falls from about 0.408 to 0.291: raising the target logit relative to the others reduces loss. The correct answer executes the requested change and gives a checkable result; the other texts do not close this calculation or trace. --- ### Question 2 Which causal order correctly connects the first three stages of “Pre-training systems and optimization”? - A. Backpropagation → Cross-entropy from logits → Causal objective - B. Cross-entropy from logits → Causal objective → Backpropagation - C. Causal objective → Backpropagation → Cross-entropy from logits - D. Causal objective → Cross-entropy from logits → Backpropagation **Answer: D.** Causal objective → Cross-entropy from logits → Backpropagation The chain follows the taught progression; reversing stages consumes a representation or state before it is produced. --- ### Question 3 If “Backpropagation” is removed, which diagnostic method is defensible? - A. Keep the same input, predict the first output that depends on “Backpropagation,” then compare the before/after trace. - B. Also change the data to amplify the difference. - C. Observe only the final output and invent the cause. - D. Conclude that the whole system fails before measuring. **Answer: A.** Keep the same input, predict the first output that depends on “Backpropagation,” then compare the before/after trace. One intervention and a prior prediction make the delta attributable to the removed mechanism. --- ### Question 4 Which verdict respects this session’s validity boundary? - A. The mechanism guarantees accuracy, speed, and stability for every workload. - B. Distributed performance depends on hardware, network, model size, and implementation; no lab estimate is a benchmark. - C. One successful example proves the whole architecture is superior. - D. The mechanism name alone is enough for a production choice. **Answer: B.** Distributed performance depends on hardware, network, model size, and implementation; no lab estimate is a benchmark. The correct answer bounds the conclusion; the others turn a local relation into a global guarantee. --- ### Question 5 Which evidence best matches the stated status of “Pre-training systems and optimization”? - A. The route loads without an error. - B. Every learner opened the file. - C. Established mechanisms; numerical simplifications are pedagogical. - D. The same result is assumed on every hardware target. **Answer: C.** Established mechanisms; numerical simplifications are pedagogical. Product or mechanism evidence must remain attributed and measured; availability and completion do not prove value. --- ### Question 6 When should a simpler baseline be preferred to “Precision and parallelism”? - A. Never: the newest mechanism wins by default. - B. As soon as one memory metric falls, regardless of quality. - C. As soon as the diagram contains fewer components. - D. When a controlled test shows equivalent quality with lower memory, latency, or complexity. **Answer: D.** When a controlled test shows equivalent quality with lower memory, latency, or complexity. The choice depends on a measured trade-off on the real workload, not novelty or one isolated metric. --- ### Question 7 A learner gets the right result but cannot explain “Cross-entropy from logits.” Which remediation is most useful? - A. Rebuild the first missing transformation, label its inputs and outputs, then test a neighboring case. - B. Accept the answer because the final number is correct. - C. Provide the final result a second time. - D. Change several variables and ask for an intuition. **Answer: A.** Rebuild the first missing transformation, label its inputs and outputs, then test a neighboring case. The remediation targets the first causal break and then requires transfer instead of rewarding a guessed result. --- ### Question 8 Which submission actually demonstrates the outcome “Explain reliable checkpoint recovery.”? - A. A list of terms without causal relations. - B. A trace with starting data, transformations, observed result, boundary, and next experiment. - C. A screenshot without values or interpretation. - D. A confident claim without a baseline or threshold. **Answer: B.** A trace with starting data, transformations, observed result, boundary, and next experiment. The correct submission makes the reasoning reproducible and the verdict revisable by future measurement.