Downloads · 30 days
0
qwen35-4b-plt
KokosDev/qwen35-4b-plt
qwen35-4b-plt is a machine learning model from KokosDev. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as mit.
Per-layer PLT checkpoints for Qwen/Qwen3.5-4B trained as same-layer MLP transcoders.
Access
Public
Updated Mar 30, 2026
Repo size
10.4 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt10.4 GB · 100%
At a glance
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Mar 30, 2026
- Updated
- Mar 30, 2026
- SHA
- 4921ae90
Base models
Model card
From the Hugging Face model README
Qwen3.5-4B PLTs
Per-layer PLT checkpoints for Qwen/Qwen3.5-4B trained as same-layer MLP transcoders.
Contents
transcoder_L0.pttranscoder_L2.pt...transcoder_L31.ptlayer_metrics.json
Layer L1 is currently missing because that checkpoint was not preserved in the training artifacts.
Training recipe
- Base model:
Qwen/Qwen3.5-4B - Objective: same-layer MLP input to same-layer MLP output
- Feature width:
16384 - Activation regime:
ReLU + L1with ghost gradients and dead-feature resampling - Training corpus:
20kimage-text subset built from CC3M and CC12M
Intended use
These checkpoints are intended for research on sparse feature discovery, circuit analysis, and PLT-based steering experiments in Qwen3.5-4B.
Notes
Checkpoint quality is not uniform across layers. Later layers generally behaved better than early layers in our experiments.
- Library
- pytorch
- License
- mit
- Created
- Mar 30, 2026
- Updated
- Mar 30, 2026