Downloads · 30 days
179
22% of all-time downloads
sae-llama-3.1-8b-64x
EleutherAI/sae-llama-3.1-8b-64x
sae-llama-3.1-8b-64x is a machine learning model from EleutherAI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a set of sparse autoencoders (SAEs) trained on Llama 3.1 8B using the 10B sample of the RedPajama v2 corpus, which comes out to roughly 8.5B tokens using the Llama 3 tokenizer. The SAEs are organized by hookpo…
All-time downloads
797
Public
Repo size
34.4 GB
Likes
17
Public
Repo makeup
Click a slice to open those files.
.safetensors34.4 GB · 100%
At a glance
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Aug 27, 2024
- Updated
- Jul 22, 2025
- SHA
- a4e880bb
Model card
From the Hugging Face model README
This is a set of sparse autoencoders (SAEs) trained on Llama 3.1 8B using the 10B sample of the RedPajama v2 corpus, which comes out to roughly 8.5B tokens using the Llama 3 tokenizer. The SAEs are organized by hookpoint, and can be loaded using the EleutherAI sparsify library.
Unlike EleutherAI/sae-llama-3.1-8b-32x, these SAEs were trained with the MultiTopK loss, which allows them to be used at varying sparsity levels at inference time. For more information, see OpenAI's description of the loss in this paper.
With the sparsify library installed, you can access an SAE like this:
from sparsify import Sae
sae = Sae.load_from_hub("EleutherAI/sae-llama-3.1-8b-64x", hookpoint="layers.23.mlp")
- Library
- transformers
- License
- mit
- Languages
- en
- Created
- Aug 27, 2024
- Updated
- Jul 22, 2025