Downloads · 30 days
0
hebbian-120m-featdim256-fast-20k
robgarct/hebbian-120m-featdim256-fast-20k
hebbian-120m-featdim256-fast-20k is a machine learning model from robgarct. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for recurrent-recall-circuits.
120M Hebbian mixer LM (onlinehebbianv10, featuredim 256), 20k steps of Pile pretraining. Stage-1 source for the factored-hybrid warm-start arms.
Access
Public
Updated Sep 7, 2026
Repo size
514 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.ckpt514 MB · 100%
At a glance
- Library
- recurrent-recall-circuits
- Access
- Public
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026
- SHA
- 4cbf2ef8
Model card
From the Hugging Face model README
hebbian-120m-featdim256-fast-20k
120M Hebbian mixer LM (online_hebbian_v10, feature_dim 256), 20k steps of
Pile pretraining. Stage-1 source for the factored-hybrid warm-start arms.
- experiment config:
lm_120m/hebbian_featdim256 - consumed by:
lm_120m/hybrid_k256_d1024 - checkpoint file:
final.ckpt(weights only; optimizer state stripped)
python experiments/language_modeling/train.py experiment=lm_120m/hybrid_k256_d1024
Resolution order for the warm start is $HYBRID_WARMSTART_CKPT, else this repo.
- Library
- recurrent-recall-circuits
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026