Downloads · 30 days
0
sae-gemma2-2b-layer12-v9c
senator1/sae-gemma2-2b-layer12-v9c
sae-gemma2-2b-layer12-v9c is a machine learning model from senator1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for sae_lens. The card lists the license as mit.
TopK Sparse Autoencoder trained on the residual stream after layer 12 of google/gemma-2-2b. Used in the LessWrong/AF post A sparse-feature audit of induction in Gemma-2-2B: GitHub · interactive dashboard.
Access
Public
Updated May 18, 2026
Repo size
302 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors302 MB · 100%
At a glance
- Library
- sae_lens
- License
- mit
- Access
- Public
- Created
- May 18, 2026
- Updated
- May 18, 2026
- SHA
- ff681761
Base models
Spaces
Model card
From the Hugging Face model README
SAE: Gemma-2-2B Layer 12 Residual Stream (v9c)
TopK Sparse Autoencoder trained on the residual stream after layer 12 of google/gemma-2-2b.
Used in the LessWrong/AF post A sparse-feature audit of induction in Gemma-2-2B:
GitHub ·
interactive dashboard.
Quick facts
| Architecture | TopK SAE |
| Hook | blocks.12.hook_resid_post |
d_in | 2,304 |
d_sae | 16,384 |
| L0 / k | 100 |
| Training tokens | 200M |
| Dataset | monology/pile-uncopyrighted (BOS-excluded) |
| Library | saprmarks/dictionary_learning 0.1.0; converted to SAELens 6.43.0 format |
| Final explained variance | 0.85 (peak 0.893) |
| Dead features | 0 |
| Hardware | Single RTX 5070 Ti (16 GB) |
Loading
from sae_lens.saes.sae import SAE
sae = SAE.load_from_disk(
"sohumsen/sae-gemma2-2b-layer12-v9c", # downloads from HF
device="cuda",
)
Or download files manually with huggingface_hub.snapshot_download and pass the local
path to SAE.load_from_disk.
What this SAE is for
It decomposes Gemma-2-2B's layer-12 residual stream into 16,384 named, monosemantic
features. Of those, ~100 are causally implicated in induction-style in-context learning
(predicting B after seeing A B ... A). The top induction feature, F15289, fires
on the second occurrence of a repeated word ("Never...Never", "Tier...Tier", ...).
For the full story — feature ranking, head-correspondence ablations, library-comparison notes (SAELens TopK plateaus on this task; dictionary_learning does not) — see the GitHub repo.
License
MIT, same as the source repository.
- Library
- sae_lens
- License
- mit
- Created
- May 18, 2026
- Updated
- May 18, 2026