Downloads · 30 days
0
sae-gpt2-medium-pile1k
masonwyatt23/sae-gpt2-medium-pile1k
sae-gpt2-medium-pile1k is a machine learning model from masonwyatt23. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for mech-interpretability. The card lists the license as other.
Top-K SAE on gpt2-medium layer-12 residual stream (blocks.12.hookresidpre, dmodel=1024). Trained on 1000 documents from the Pile to demonstrate that the platform can train a usable SAE at non-toy scale on a single 128…
Access
Public
Updated May 28, 2026
Repo size
16.8 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors16.8 MB · 96%
At a glance
- Library
- mech-interpretability
- License
- other
- Access
- Public
- Created
- May 28, 2026
- Updated
- May 28, 2026
- SHA
- a6f9ee5e
Model card
From the Hugging Face model README
sae-run51-gpt2-medium
Top-K SAE on gpt2-medium layer-12 residual stream (blocks.12.hook_resid_pre, d_model=1024). Trained on 1000 documents from the Pile to demonstrate that the platform can train a usable SAE at non-toy scale on a single 128 GB Apple Silicon MBP. 2048 features with k=32 gives a ~64x expansion factor and roughly 3% feature density, which is typical for mid-network SAEs on GPT-2 class models.
Model card
| Field | Value |
|---|---|
| Model | gpt2-medium |
| Hook site | blocks.12.hook_resid_pre |
| Features | 2048 (k=32) |
| Live features | 473.0 |
| Explained variance | 0.9748 |
| Training tokens | 8125.0 |
| Source run | run-000051 if known |
How to use
# Install
pip install mech-interpretability # or: uv sync --extra interp
from mech_interp.sae.registry import load_pretrained_sae
sae, config = load_pretrained_sae("sae-run51-gpt2-medium")
Or via the CLI:
mech load-sae --name sae-run51-gpt2-medium
Provenance
- Source run:
run-000051(seeenvironment.jsonin this repo) - Python
3.12.2, torch2.12.0 - Platform:
masonwyatt23/sae-gpt2-medium-pile1k
Files
| File | Description |
|---|---|
sae_weights.safetensors | Encoder + decoder weights |
sae_config.json | Hyperparameters and training metadata |
feature_analysis.json | Per-feature top-activating prompts |
environment.json | Full environment provenance snapshot |
bundle_metadata.json | Machine-readable bundle manifest |
- Library
- mech-interpretability
- License
- other
- Created
- May 28, 2026
- Updated
- May 28, 2026