Downloads · 30 days
0
sae-qwen2.5-1.5B-res
HuggingAnalist/sae-qwen2.5-1.5B-res
sae-qwen2.5-1.5B-res is a machine learning model from HuggingAnalist. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Sparse Autoencoders trained on the residual stream of Qwen/Qwen2.5-1.5B-Instruct for layers 17–21 except 20.
Access
Public
Updated May 29, 2026
Repo size
1.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.2 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- May 29, 2026
- Updated
- May 29, 2026
- SHA
- 2a9297bf
Base models
Model card
From the Hugging Face model README
Qwen2.5-1.5B-Instruct Sparse Autoencoders
Sparse Autoencoders trained on the residual stream of Qwen/Qwen2.5-1.5B-Instruct for layers 17–21 except 20.
Training Details
- Architecture: Standard SAE (L1 penalty)
- Hook:
model.layers.{layer}(residual stream post-layer) - d_model: 1536
- Expansion Factor: 16× → 24,576 features
- Training Tokens: 100M
- Dataset: Mixed reasoning (NuminaMath-CoT) + safety pairs (Anthropic HH-RLHF) + jailbreak prompts (JailbreakBench) + general knowledge (FineWeb-Edu)
Repository Structure
layer_17/final_sae/ ← SAE weights + config for layer 17
...
layer_21/final_sae/
Usage
from sae_lens import SAE
sae, cfg_dict, sparsity = SAE.from_pretrained(
"HuggingAnalist/sae-qwen2.5-1.5B-res",
subfolder="layer_17/final_sae",
)
- License
- apache-2.0
- Created
- May 29, 2026
- Updated
- May 29, 2026