# Teacher guide — Gated DeltaNet and selective forgetting

**Duration:** 120 minutes<br>
**Positioning:** Add decay and write gates to correctable memory.<br>
**Expected evidence:** Established: the delta rule, exponential decay, and the half-lives computed here. Pedagogical: the α, β values and the 2×2 matrices of the trace. Reported: the exact gate choices of named models.

## Observable outcomes and preparation

- Compute decay.
- Interpret a write gate.
- Compare with state-space principles.

Before class, the instructor runs the worked case and lab, prints the exercise packet, prepares a four-column board—assumption, prediction, observation, delta—and checks that every mathematical notation is paired with dimensions. The demonstration must not become slide reading.

## Diagnostic

1. Explain in one sentence: **Why forget**. What observation would falsify your explanation?
2. Explain in one sentence: **Decay gate**. What observation would falsify your explanation?
3. Explain in one sentence: **Write gate**. What observation would falsify your explanation?

**Teaching decision:** if two of three answers remain nominal or lack a validity condition, rebuild the vocabulary with a numerical example before any formula. A fluent but unfalsifiable answer does not count as mastery.

## Timed plan

| Time | Activity | Observable evidence |
|---|---|---|
| 0–10 min | Individual diagnostic, then pair comparison | Three answers and one named uncertainty |
| 10–25 min | Situation and vocabulary | Annotated input → state → output diagram |
| 25–55 min | Develop the mechanism on the board | Shapes, assumptions, and intermediate calculation visible |
| 55–75 min | Worked case with deliberate errors | Reasoned correction, not only the right number |
| 75–95 min | Causal lab: predict, change one variable, run | Prediction / observation / delta table |
| 95–112 min | Exercises 1 and 2 with peer correction | Retained artifact and applied rubric |
| 112–120 min | Exit ticket and transfer | Mechanism, boundary, next experiment |

## Teaching notes

# Chapter 11 — Forgetting: Gated DeltaNet and the Mamba connection

### 11.1 Why forgetting helps

**Goal:** prevent old information from occupying memory forever. A gate is a learned number, often between 0 and 1, that controls how much information passes.

**Intuition:** A whiteboard is useful because we can partly erase outdated notes before writing new ones.

**Step by step**

- Apply broad decay: S ← αS, where α is near 1 to remember and near 0 to forget strongly.

- Apply the targeted delta correction for the current key.

- Use β to control write strength.

**Worked example:** if S contains 10 and α=0.8, decay leaves 8 before the new write. Repeating 0.8 decay gives 10, 8, 6.4, 5.12… Old influence fades smoothly.

**Why it matters:** Forgetting is selective resource management. It can improve effective capacity when old details are no longer useful.

**Quick check:** Does α=1 erase memory? Answer: No; it keeps the old state unchanged before the new write.

### 11.2 Connection to Mamba

**Goal:** place gated recurrent memory in a broader family. Mamba is a selective state-space model: it carries a fixed-size state through sequence time and lets input-dependent controls decide what to retain and inject.

**Intuition:** Both systems resemble a running scientific experiment: keep a state, decay or transform it, add new evidence, and produce an output.

**Step by step**

- Shared theme: recurrent fixed-size state.

- Shared theme: learned, input-dependent gates.

- Important difference: Delta-style memory explicitly uses key-value association and correction; Mamba is usually written with state-space equations.

**Worked example:** The connection is conceptual, not an assertion that the two architectures are identical.

**Why it matters:** Recognizing families helps us compare design choices without collapsing distinct mechanisms into one name.

**Quick check:** What is the shared central idea? Answer: a learned fixed-size state updated as the sequence advances.

### Running the worked case

With α=0.8, a trace of value 1 becomes 0.8 then 0.64 without new writes. Correction β=0.5 then adds only half the observed error.

Do not reveal the result at once. Ask learners to predict the next operation, its shape, and the expected sign. After each line ask: “What changed? What stayed fixed? Which assumption did we use?” A calculation error repaired with a causal chain is worth more than a guessed result.

### Lab protocol

1. Write a qualitative and, where possible, numerical prediction before touching a control.
2. Change one variable only; retain a capture or record initial and final values.
3. Explain the delta through the mechanism, not “the tool did that.”
4. Test one boundary value and state where the model stops representing a real system.

## Misconceptions

| # | Observable misconception | Grounded correction | Probe |
|---|---|---|---|
| 1 | “Why forget guarantees the outcome without assumptions or measurement.” | Reverse the reflex: forgetting is a function, not a failure. A fixed-capacity memory that never frees anything eventually distinguishes nothing; erasing the stale is exactly what keeps the present readable. | Ask for a counterexample, then restate the mechanism with its validity condition. |
| 2 | “Decay gate guarantees the outcome without assumptions or measurement.” | A gate α_t ∈ [0,1], produced at every token, multiplies the state before the write: S′ = α_t·S_{t−1}. At 0.8 a trace follows 1 → 0.8 → 0.64, and its half-life is ln(0.5)/ln(0.8) ≈ 3.1 tokens. The model learns when to tighten or open the gate. | Ask for a counterexample, then restate the mechanism with its validity condition. |
| 3 | “Write gate guarantees the outcome without assumptions or measurement.” | β_t doses the delta correction: S_t = S′ + β_t k eᵀ. In the trace, β = 0.5 applies only half the error — exactly (1−β)·e remains. β ≈ 0 ignores, β = 1 fully corrects, decided token by token. | Ask for a counterexample, then restate the mechanism with its validity condition. |

> **Boundary to maintain:** The Mamba parallel is conceptual. It does not imply Gated DeltaNet and Mamba are interchangeable.

## Probing questions

1. If we remove or reverse **Why forget**, which output changes first, and what observation would show it?
2. If we remove or reverse **Decay gate**, which output changes first, and what observation would show it?
3. If we remove or reverse **Write gate**, which output changes first, and what observation would show it?
4. If we remove or reverse **Stability**, which output changes first, and what observation would show it?
5. If we remove or reverse **Effective capacity**, which output changes first, and what observation would show it?
6. If we remove or reverse **Connection to Mamba**, which output changes first, and what observation would show it?

## Assessment

| Level | Criterion |
|---|---|
| 0 | Repeats terms without connecting input, transformation, and output. |
| 1 | Describes the chain but checks neither shape nor assumption. |
| 2 | Executes the case, explains the result, and names one limitation. |
| 3 | Transfers to a new case, compares an alternative, and proposes a measurement that could invalidate the choice. |

**Exit threshold:** level 2 on the worked case and at least one exercise; a memorized formula without interpretation remains level 1.

## Observation and remediation protocol

During discussion, the instructor records evidence rather than impressions. Evidence of understanding contains a named object, a justified transformation, and a checkable consequence. If a learner gives the right result without a chain, ask for the preceding line. If the chain is coherent but the result is wrong, preserve the chain and isolate the arithmetic error. If vocabulary from another concept is used, compare both mechanisms in an input, state, output, cost, and boundary table. Remediation targets the first break only: vocabulary, shapes, operation, interpretation, or claim scope. After correction, use a neighboring case with a changed value; success on the same example does not prove transfer. For pair work, assign operator and verifier roles, then swap. The verifier does not supply the answer: they request an assumption, check the shape, and ask what observation could contradict the reasoning. The instructor retains the exit ticket and classifies the dominant break. The next session opens with a three-minute problem aimed at that break instead of repeating the whole lesson.

## Differentiation

- **Support:** provide shapes and the first transformation; let the learner complete interpretation and boundary.
- **Core path:** worked case without result, lab with one assigned variable, diagnostic exercise.
- **Extension:** change one assumption, compare two mechanisms, and define the metric that would decide between them.

## Post-session follow-up

Within twenty-four hours, return each annotated exit ticket with one priority, the exact resource to reopen, and a mini-case different from the worked case. Revision requires three items: a written prediction, a retained trace, and one sentence explaining the delta. At the next session, sample two submissions: one that repaired the break and one that remains ambiguous. Discuss them anonymously, then state the criterion that separates them. Do not use completion rate as evidence of mastery. Evidence of remediation is a correct chain on a new case with a stated boundary. If the same break appears in more than one third of the group, repair the support or demonstration before blaming learners.

Follow-up closes only when the new artifact shows the causal chain, check, and boundary—not merely when a file has been submitted.

## Sources and evidence boundary

- Owner-supplied bilingual course packet, Chapter 11.
- Yang, Kautz & Hatamizadeh, “Gated Delta Networks: Improving Mamba2 with Delta Rule”, ICLR (2025), arXiv:2412.06464.
- Gu & Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”, arXiv:2312.00752 (2023).
- Course source packet supplied by the owner; named-product details remain source-reported until primary verification.

> **Scope:** Established: the delta rule, exponential decay, and the half-lives computed here. Pedagogical: the α, β values and the 2×2 matrices of the trace. Reported: the exact gate choices of named models. These references support the session frame; they do not turn a reported product choice into an independently verified result.

## Exit ticket

In no more than six lines: mechanism; calculation or trace; observation; boundary; evidence level; next experiment. The instructor marks one priority causal break for revision.