# Exercises with solutions — Gated DeltaNet and selective forgetting
**General instruction:** every answer must show data, transformation, result, one check, and one limitation. A bare number or copied definition is insufficient.
> **Starting data:** With α=0.8, a trace of value 1 becomes 0.8 then 0.64 without new writes. Correction β=0.5 then adds only half the observed error.
>
> **Boundary to retain:** The Mamba parallel is conceptual. It does not imply Gated DeltaNet and Mamba are interchangeable.
## Exercise 1 — Calculated trace — Why forget
Reproduce and annotate the chain `S_t=α_t S_{t−1}+update_t`. Increase α from 0.8 to 0.96 for an initial trace of 1 with no new write. Calculate the next two states and compare retention.
**Deliverable:** a data → operation → result → interpretation table, plus two sentences about the changed value.
Worked solution
With α=0.8, a trace of value 1 becomes 0.8 then 0.64 without new writes. Correction β=0.5 then adds only half the observed error.
**Solved variant:** With α=0.8 the trace becomes 0.8 then 0.64. With α=0.96 it becomes 0.96 then 0.9216. Retention rises sharply, but α near 1 also slows forgetting of stale information.
Reverse the reflex: forgetting is a function, not a failure. A fixed-capacity memory that never frees anything eventually distinguishes nothing; erasing the stale is exactly what keeps the present readable. A gate α_t ∈ [0,1], produced at every token, multiplies the state before the write: S′ = α_t·S_{t−1}. At 0.8 a trace follows 1 → 0.8 → 0.64, and its half-life is ln(0.5)/ln(0.8) ≈ 3.1 tokens. The model learns when to tighten or open the gate. The minimum check covers dimensions, sign, and order of magnitude. If the observed change contradicts the prediction, locate the first operation whose direction changes instead of fixing only the final line.
Reasoned solution
β_t doses the delta correction: S_t = S′ + β_t k eᵀ. In the trace, β = 0.5 applies only half the error — exactly (1−β)·e remains. β ≈ 0 ignores, β = 1 fully corrects, decided token by token. Stability rests on binding details: α bounded in [0,1] — not merely |α| ≤ 1, a negative α would flip the sign each step —, normalized keys, and a declared operation order: decay, read, correct. The Mamba parallel is conceptual. It does not imply Gated DeltaNet and Mamba are interchangeable.
The claim mixes a local relation with a global guarantee. A defensible version states only the observed mechanism, test conditions, and measured metric. Stop the test if shapes become invalid, the metric crosses the declared degradation threshold, or another variable changes.
Elements of a strong solution
The right quantity is useful lifetime: half-life ≈ 69 tokens at α = 0.99, ≈ 3.1 at α = 0.80, 1.0 at α = 0.5. Effective capacity combines S’s size, forgetting, and key separation — not the advertised maximum context length. The overlap is real: recurrence plus learned controls. So is the difference: the delta memory READS its state (v̂ = S′ᵀk) and writes the error; a classical SSM filters the signal with no associative read. Same family of motivations, distinct equations. Established: the delta rule, exponential decay, and the half-lives computed here. Pedagogical: the α, β values and the 2×2 matrices of the trace. Reported: the exact gate choices of named models.
A strong answer does not make the newer mechanism the default winner. It retains a measurable baseline, sets thresholds before testing, and separates component cost from whole-system behavior. The verdict names what remains uncertain and the next evidence that could change it.