Downloads · 30 days
0
hebbian-120m-featdim16-fast-20k
robgarct/hebbian-120m-featdim16-fast-20k
hebbian-120m-featdim16-fast-20k is a machine learning model from robgarct. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for recurrent-recall-circuits.
120M Hebbian mixer LM (onlinehebbianv10, featuredim 16), 20k steps of Pile pretraining. Stage-1 source for the factored-hybrid warm-start arms.
Access
Public
Updated Sep 3, 2026
Repo size
496 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.ckpt496 MB · 100%
At a glance
- Library
- recurrent-recall-circuits
- Access
- Public
- Created
- Sep 3, 2026
- Updated
- Sep 3, 2026
- SHA
- f08e53bf
Model card
From the Hugging Face model README
hebbian-120m-featdim16-fast-20k
120M Hebbian mixer LM (online_hebbian_v10, feature_dim 16), 20k steps of
Pile pretraining. Stage-1 source for the factored-hybrid warm-start arms.
- experiment config:
lm_120m/hebbian_featdim16 - consumed by:
lm_120m/hybrid_k16_d256 - checkpoint file:
final.ckpt(weights only; optimizer state stripped)
python experiments/language_modeling/train.py experiment=lm_120m/hybrid_k16_d256
Resolution order for the warm start is $HYBRID_WARMSTART_CKPT, else this repo.
- Library
- recurrent-recall-circuits
- Created
- Sep 3, 2026
- Updated
- Sep 3, 2026