history-compressor-pocket

History Compressor Pocket

History Compressor Pocket is a controlled, modernized experiment inspired by the event-driven timescales of neural history compressors. A slow GRU level updates only when an informative control symbol arrives; a wider plain tanh RNN updates on every random distractor. Both models have exactly 3,464 parameters.

Training mixes distractor gaps from 8 to 32 steps. Evaluation extends the gap to 64 and 128 steps without further optimization. The interactive Space exposes the hidden-state trajectories and every boundary prediction.

Verified result

Across three independent seeds, the 3,464-parameter event-gated model achieved 100% boundary accuracy at gaps 32, 64, and 128. The exactly parameter-matched plain RNN averaged 58.17%, 58.40%, and 58.50%, respectively.

The control was notably unstable rather than uniformly incapable: its three gap-128 runs reached 50.33%, 100%, and 25.16%. Event gating therefore improved training reliability and guaranteed invariance to the distractor duration in this synthetic benchmark.

uv run python projects/history-compressor-pocket/train.py
uv run pytest tests/test_history_compressor_pocket.py