Downloads · 30 days
0
matrix-sae
JackYoung27/matrix-sae
matrix-sae is a machine learning model from JackYoung27. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as mit.
Sparse autoencoder checkpoints for matrix-valued recurrent states. 25 checkpoints, 0.6 GB. Code and paper: github.com/JackYoung27/matrix-sae.
Access
Public
Updated Apr 16, 2026
Repo size
642 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt642 MB · 100%
At a glance
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Apr 16, 2026
- Updated
- Apr 16, 2026
- SHA
- 3a3f5bd8
Base models
Model card
From the Hugging Face model README
matrix-sae
Sparse autoencoder checkpoints for matrix-valued recurrent states. 25 checkpoints, 0.6 GB. Code and paper: github.com/JackYoung27/matrix-sae.
| group | n | size | types | description |
|---|---|---|---|---|
encoder_swap_checkpoints | 2 | 132 MB | bilinear_flat, bilinear_tied | Five-variant encoder-swap across three seeds. |
higher_rank | 2 | 16 MB | mixed | Rank-2 and rank-4 bilinear at layers 9 and 17. |
qwen3_5-0_8b_sl1024_ns2000 | 18 | 452 MB | bilinear, flat, rank1 | Main 0.8B pack: layer 9 per-head flat, rank1, bilinear. |
review_downstream_sensitivity | 2 | 8 MB | bilinear | k and nf sweeps. |
sparsity_compare | 1 | 4 MB | mixed | TopK versus BatchTopK for flat and bilinear. |
Tags: {sae_type}_L{layer}_H{head}_nf{n_features}_k{k}_s{seed}.
from loader import load_checkpoint
model, cfg, entry, _ = load_checkpoint('qwen3_5-0_8b_sl1024_ns2000/bilinear_L9_H0_nf2048_k32_s42')
- Library
- pytorch
- License
- mit
- Created
- Apr 16, 2026
- Updated
- Apr 16, 2026