memory-tape-pocket

Memory Tape Pocket

Memory Tape Pocket is a compact differentiable-memory retest inspired by the content-addressing mechanism of Neural Turing Machines. It learns random key-value associative recall on tapes containing two to eight slots, then faces unseen tapes with 16 and 32 slots.

The control is a larger fixed-state GRU trained on the same batches. The interactive Space exposes the complete external tape and the learned read weight assigned to every slot.

Verified result

Across three independent training seeds, the 4,673-parameter content-addressed model achieved 100% exact recall on 8-, 16-, and 32-slot tapes. At 32 slots, four times the maximum training length, its read head placed 99.974% of its attention on the correct slot.

The larger 5,584-parameter fixed-state GRU reached 13.51% accuracy at eight slots, 7.66% at 16 slots, and 4.60% at 32 slots. This benchmark isolates the inductive bias of external content addressing; it does not claim the tiny model implements every component of a full Neural Turing Machine.

uv run python projects/memory-tape-pocket/train.py
uv run pytest tests/test_memory_tape_pocket.py