Downloads · 30 days
0
andyrdt/saes-llama-3.1-8b-instruct
saes-llama-3.1-8b-instruct is a machine learning model from andyrdt. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Residual stream SAEs for Llama-3.1-8B-Instruct.
Downloads · 30 days
0
Access
Public
Updated May 21, 2025
Repo size
120 GB
Likes
8
Public
Click a slice to open those files.
.pt120 GB · 100%
From the Hugging Face model README
Residual stream SAEs for Llama-3.1-8B-Instruct.
These SAEs were trained using a blend of chat (lmsys/lmsys-chat-1m) and pretraining data (monology/pile-uncopyrighted), and also a small amount of emergent misalignment data.
Each SAE is trained using BatchTopK. For each layer, we train 4 SAEs, with k=32,64,128,256.
For more training details, see https://github.com/andyrdt/dictionary_learning/tree/andyrdt/llama_saes.