Downloads · 30 days
0
transcoder-TinyStories-3M-4x
abmakkeh/transcoder-TinyStories-3M-4x
transcoder-TinyStories-3M-4x is a machine learning model from abmakkeh. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is a set of transcoders trained on the TinyStories-3M using the 2.2M sample of the TinyStories dataset, which comes out to roughly 2.2B tokens using the full GPT-Neo tokenizer. We trained the transcoders on the M…
Access
Public
Updated Dec 21, 2025
Repo size
1.6 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.6 MB · 100%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Dec 21, 2025
- Updated
- Dec 21, 2025
- SHA
- cb235f2e
Base models
Datasets
Model card
From the Hugging Face model README
This is a set of transcoders trained on the TinyStories-3M using the 2.2M sample of the TinyStories dataset, which comes out to roughly 2.2B tokens using the full GPT-Neo tokenizer. We trained the transcoders on the MLP layers 2, 4, and 7 with a dictionary size 4-times that of the MLP layer. The transcoders are organized by hookpoint.
Usage
With the sae library installed, you can access an SAE like this:
from sae import Sae
sae = Sae.load_from_hub("abmakkeh/transcoder-TinyStories-3M-4x", hookpoint="h.1.mlp")
- Library
- transformers
- License
- apache-2.0
- Languages
- en
- Created
- Dec 21, 2025
- Updated
- Dec 21, 2025