Downloads · 30 days
31
27% of all-time downloads
esm2-sae-topk-16384-k512
aasatorres/esm2-sae-topk-16384-k512
esm2-sae-topk-16384-k512 is a machine learning model from aasatorres. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as cc-by-4.0.
Sparse Autoencoder (SAE) trained on residue-level embeddings from ESM-2 (650M, layer 33) for interpretability research on protein language models.
All-time downloads
114
Public
Repo size
504 MB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.pt504 MB · 100%
At a glance
- License
- cc-by-4.0
- Access
- Public
- Created
- May 25, 2026
- Updated
- May 25, 2026
- SHA
- aa9433d4
Model card
From the Hugging Face model README
ESM-2 Sparse Autoencoder (TopK, K=16384, k=512)
Sparse Autoencoder (SAE) trained on residue-level embeddings from ESM-2 (650M, layer 33) for interpretability research on protein language models.
Model details
| Parameter | Value |
|---|---|
| Architecture | TopK SAE |
| Input dim | 1280 (ESM-2 layer 33) |
| Latent dim | 16384 |
| TopK k | 512 |
| Training proteins | 50,000 Swiss-Prot |
| ESM-2 model | facebook/esm2_t33_650M_UR50D |
Files
best.pt: model weights (PyTorch)config.json: architecture config and input normalization statistics (emb_mean,emb_std)metrics.csv: training and validation metrics per epoch
Usage
import torch, json
from pathlib import Path
config = json.load(open("config.json"))
checkpoint = torch.load("best.pt", map_location="cpu")
# checkpoint["model_state_dict"] contains the SAE weights
Companion datasets
- Processed dataset: https://doi.org/10.5281/zenodo.20379180
- Alignment scores & logs: https://doi.org/10.5281/zenodo.20384152
- Code: https://github.com/Andre-Satorres/PLM-Interpretability
- License
- cc-by-4.0
- Created
- May 25, 2026
- Updated
- May 25, 2026