Downloads · 30 days
0
HobbyLM-SAE
rootxhacker/HobbyLM-SAE
HobbyLM-SAE is a machine learning model from rootxhacker. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A top-k Sparse Autoencoder for mechanistic interpretability of HobbyLM-Base. It decomposes the residual stream after layer 8 into a sparse, overcomplete dictionary of 12288 features (32 active per token), most of them…
Access
Public
Updated Jun 21, 2026
Repo size
75.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors75.6 MB · 98%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jun 21, 2026
- Updated
- Jun 21, 2026
- SHA
- 3f61a99b
Model card
From the Hugging Face model README
HobbyLM-SAE
A top-k Sparse Autoencoder for mechanistic interpretability of HobbyLM-Base. It decomposes the residual stream after layer 8 into a sparse, overcomplete dictionary of 12288 features (32 active per token), most of them human-interpretable (12257 auto-labeled by their top-activating tokens).
Files
sae.safetensors— the SAE weights (W_enc,W_dec,b_enc,b_dec).labels.json— per-feature auto-derived label + example top-activating tokens.meta.json— layer, activation scale, base-model run, and SAE config.
Reconstructs ~97% of the activation variance at L0=32. Reference code + training harness:
https://github.com/harishsg993010/HobbyLM (hobbylm/sae.py, training/modal_sae.py). Apache-2.0.
- License
- apache-2.0
- Created
- Jun 21, 2026
- Updated
- Jun 21, 2026