1. Limit of addition

The problem: Session 13, last write: k=[1,0] reused, read [11,12] — neither [2,3] nor [9,9]. An additive memory only knows how to reinforce: updating an association (“the price moved to 9”) yields a blend of both versions instead of a replacement.

The idea: The diagnosis is structural: S ← S + k vᵀ never consults S. The write is decided without knowing what memory already holds — repeating information amplifies it, correcting it entangles it.

Why / at what price: Naming the cause — the blind write — points to the remedy: read before writing. Addition’s remaining virtue is its simplicity: one operation, no state consulted; everything the fix adds will be paid for in per-token compute.

2. Read before writing

The problem: To correct without blending, you must know what memory would answer BEFORE writing. Otherwise there is no way to tell “new information” (write hard) from “already known” (do nothing).

The idea: Memory predicts first: v̂ = Sᵀk. On S = [[2,3],[5,1]] with k=[1,0], v̂ = [2,3] — exactly what an aligned query would read. The write becomes conditional on the state, no longer only on the input.

v̂=Sᵀk

Why / at what price: That read is what makes correction possible — and it is one more multiplication per token, before anything is even written. The price: v̂ is only reliable if k aligns with what was written; a misplaced key “corrects” a prediction that never existed.

3. Local error

The problem: The read gives v̂ = [2,3], the target is v = [9,9]. Writing all of v would superpose again — straight back to session 13’s problem. What exactly must be written to get from one to the other?

The idea: Only the difference: e = v − v̂ = [7,6]. It carries both what is missing (positive components) and what must be erased (negative ones). And if v̂ = v, then e = [0,0]: the redundant token writes nothing.

e=v−v̂

Why / at what price: The zero error is the first economy mechanism: no useless rewriting, no drift on repetitions. The price: e is local to the current key — it corrects what k reads, not the whole memory; an error seen through the wrong key stays invisible.

4. Delta update

The problem: e says what to correct; it must still land in the right place at the right strength. A correction spread everywhere would degrade the other rows; one that is too strong overshoots — case D reads [−5,−4].

The idea: S ← S + β k eᵀ: k localizes (row 1 only), e carries the content, β sets the dose. With β=1 and ‖k‖=1: exact replacement, [2,3] → [9,9], row 2 untouched. With β=0.5: halfway, [5.5, 6].

S←S+βk(v−Sᵀk)ᵀ

Why / at what price: After one step, (1 − β·‖k‖²) of the error remains: exact replacement needs β=1 AND a normalized key. The price: calibration — a mis-set β or ‖k‖ ≠ 1 overshoots or oscillates, like the beginner session’s learning rate… at token rate.

5. Fast weights

The problem: S changes at every token during inference — so what exactly was “learned” in training? If everything moves all the time, the beginner session’s training/conversation distinction seems to collapse.

The idea: Two speeds coexist: S is a FAST weight, rewritten token by token and discarded at sequence end; the matrices producing k, v, β are SLOW parameters, frozen at inference. Training learns how to drive the memory, not its contents.

Why / at what price: This separation reconciles the two regimes: context writes itself into S without touching parameters. The price: debugging changes in kind — odd behavior can come from state (this very sequence) or from parameters (training), and the remedies have nothing in common.

6. Orientations

The problem: You open two implementations: one writes S + βk eᵀ and reads qᵀS; the other writes S + βe kᵀ and reads Sq. The formulas differ — is one of them wrong?

The idea: Neither: the whole presentation transposes (rows ↔ columns) without changing the mechanism. What is immutable: read, compare, correct — and the DECLARED shapes: k(d_k), e(d_v), S(d_k×d_v) in one convention, transposed in the other.

Why / at what price: Knowing this keeps you from “fixing” correct code. The price: convention freedom is a team trap — mixing both in one file produces silent bugs. Hence the house rule: declare the shapes next to every formula.