Downloads · 30 days
0
sae_phi-3_mini_4k_ins_blocks_16_hook_resid_post_1B_tokens
coai/sae_phi-3_mini_4k_ins_blocks_16_hook_resid_post_1B_tokens
sae_phi-3_mini_4k_ins_blocks_16_hook_resid_post_1B_tokens is a machine learning model from coai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
Dataset: mlfoundations/dclm-baseline-1.0 Hookpoint: blocks.16.hookresidpost Layer: 16
Access
Public
Updated Aug 18, 2024
Repo size
2.4 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors2.4 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Aug 18, 2024
- Updated
- Aug 18, 2024
- SHA
- 82263c8c
Datasets
Model card
From the Hugging Face model README
Training an Sparse Autoencoder for Mechanistic Interpretability on PHI-3-mini-instruct with 1Billion Tokens
Dataset: mlfoundations/dclm-baseline-1.0
Hookpoint: blocks.16.hook_resid_post
Layer: 16
Trainingsteps 250_000
Batchsize: 4096
Context_size: 2048
ExpansionFaktor: 32
WandB Training Report: https://api.wandb.ai/links/kdt/h9edatb5
@misc{schacht2024sae4phi3,
title = {SAE for Phi-3 Mini Instruct Layer 16
author = {Sigurd Schacht},
year = {2024},
howpublished = {\url{}},
}}
- License
- mit
- Languages
- en
- Created
- Aug 18, 2024
- Updated
- Aug 18, 2024