curiosity-cabinet

Curiosity Cabinet

Curiosity Cabinet places an intrinsically motivated agent in three rooms:

• an already simple, predictable process;
• a nonlinear but learnable process;
• an irreducibly random "noisy TV."

Every room has its own tiny neural world model. One agent chooses the room with the largest recent prediction error; another chooses the room producing the greatest recent reduction in error. Repeated seeded trials measure where each agent spends its experience and how accurately it ultimately models the learnable process.

The historical anchor is Schmidhuber's 1991 A Possibility for Implementing Curiosity and Boredom in Model-Building Neural Controllers, which proposed reinforcement for actions that increase a model network's knowledge. This project is a modern noisy-TV stress test of related intrinsic rewards, not an exact reproduction of the original controller or environment.

Verified results

Each reward ran for 1,200 steps under 60 independent seeds:

Fractions cover the middle half of training, after initialization and before the final window. Prediction error reliably preferred the process that remained unpredictable. Learning progress reduced noisy-TV attention by 57.89 points and kept exploration roughly balanced. It did not learn a perfect room-selection policy, and its downstream world-model improvement was modest.

Reproduce

uv run python projects/curiosity-cabinet/train.py