o = read address (q1) · × = write address (k1) · hollow = concept prototype (shifted per head).
Faint rings are the 3×3 cell window (radius 1.5).
Cell colour — log(1+‖S‖), the Frobenius norm of that cell's
(dk+1)×dv DeltaNet state. Each head owns its own lattice (one panel per head); the selected
head is drawn bright, the others dimmed.
read window — the 9 cells a query at the
token's q-address touches: o_t = Σj wqj·S[cellj]ᵀq2.
write window — the 9 cells its key writes:
S[cell] += w·β·(v − S[cell]k2)k2ᵀ.
Addresses: a = ½G(1+tanh u), one per (token, head);
the 1D weights are the cos² bump b(t)=cos²(πt/3) on the three nearest cells
(normalised so the self-kernel is 1), giving a 3×3 window in 2D.
Dense routing floor f — mixes a uniform share into every routing weight,
w ← (1−f)w + f/9 (the reference applies the analogue
D ← (1−f)D + f to the intra-chunk scores, which are identically 1 at chunk = 1).
Raising it blurs every token's writes across its whole 3×3 window.
Reading it — tokens are synthetic concepts with fixed addresses, so writes pile into a few
cells per concept; flipping a concept's value overwrites the cell state (watch a cell's colour shift and o_t swing),
and an immediate repeat writes almost nothing (v − S k2 ≈ 0).