Downloads · 30 days
6
10% of all-time downloads
llava-micro-sae-test
AKG2/llava-micro-sae-test
llava-micro-sae-test is a machine learning model from AKG2. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
A Sparse Autoencoder trained on CLIP ViT-L/14 patch activations (Layer 23) extracted from llava-hf/llava-1.5-7b-hf.
All-time downloads
62
Public
Repo size
67.2 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt33.6 MB · 100%
At a glance
- Access
- Public
- Created
- Mar 6, 2026
- Updated
- Mar 6, 2026
- SHA
- a975a032
Model card
From the Hugging Face model README
Micro-SAE for LLaVA 1.5 7B Vision Tower
A Sparse Autoencoder trained on CLIP ViT-L/14 patch activations (Layer 23)
extracted from llava-hf/llava-1.5-7b-hf.
Architecture
- Input dim: 1024 (CLIP ViT-L/14 hidden size)
- Dictionary size: 4096 (4× expansion)
- Activation: ReLU + L1
Usage
import torch
import json
# Load config
with open("config.json") as f:
config = json.load(f)
# Recreate and load SAE
sae = SparseAutoencoder(config["input_dim"], config["dict_size"])
sae.load_state_dict(torch.load("micro_sae_1024d.pt"))
Training Data
- 20,000 COCO images
- 500 zebra images (concept-specific)
- 500 fire-truck images (concept-specific)
- Created
- Mar 6, 2026
- Updated
- Mar 6, 2026