Downloads · 30 days
0
sae-llama-3.1-8b-32x
EleutherAI/sae-llama-3.1-8b-32x
sae-llama-3.1-8b-32x is a machine learning model from EleutherAI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.
This is a set of sparse autoencoders (SAEs) trained on Llama 3.1 8B using the 10B sample of the RedPajama v2 corpus, which comes out to roughly 8.5B tokens using the Llama 3 tokenizer. The SAEs are organized by hookpo…
Access
Public
Updated Aug 6, 2024
Repo size
17.2 GB
Likes
13
Public
Repo makeup
Click a slice to open those files.
.safetensors8.6 GB · 100%
At a glance
- Library
- transformers
- License
- mit
- Access
- Public
- Created
- Aug 2, 2024
- Updated
- Aug 6, 2024
- SHA
- ed63bac9
Model card
From the Hugging Face model README
This is a set of sparse autoencoders (SAEs) trained on Llama 3.1 8B using the 10B sample of the RedPajama v2 corpus, which comes out to roughly 8.5B tokens using the Llama 3 tokenizer. The SAEs are organized by hookpoint, and can be loaded using the EleutherAI sae library.
With the sae library installed, you can access an SAE like this:
from sae import Sae
sae = Sae.load_from_hub("EleutherAI/sae-llama-3.1-8b-32x", hookpoint="layers.23.mlp")
- Library
- transformers
- License
- mit
- Languages
- en
- Created
- Aug 2, 2024
- Updated
- Aug 6, 2024