Downloads · 30 days
19
40% of all-time downloads
imagenet-sweep-vanilla-x64-Spatial_max_8-hook_resid_post-965.125-99
Prisma-Multimodal/imagenet-sweep-vanilla-x64-Spatial_max_8-hook_resid_post-965.125-99
imagenet-sweep-vanilla-x64-Spatial_max_8-hook_resid_post-965.125-99 is a machine learning model from Prisma-Multimodal. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This model is a sparse autoencoder trained on CLIP's internal representations.
All-time downloads
48
Public
Repo size
302 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt302 MB · 100%
At a glance
- Access
- Public
- Created
- Jan 29, 2025
- Updated
- Jan 29, 2025
- SHA
- 8b3af6e8
Model card
From the Hugging Face model README
CLIP Sparse Autoencoder Checkpoint
This model is a sparse autoencoder trained on CLIP's internal representations.
Model Details
Architecture
- Layer: 8
- Layer Type: hook_resid_post
- Model: open-clip:laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K
- Dictionary Size: 49152.0
- Input Dimension: 768.0
- Expansion Factor: 64.0
- CLS Token Only: False
Training
- Training Images: 1299936.0000
- Learning Rate: 0.0124
- L1 Coefficient: 0.0000
- Batch Size: 4096.0
- Context Size: 49.0
Performance Metrics
Sparsity
-
L0 (Active Features): 965.1250
-
Dead Features: 0.0000
-
Mean Passes Since Fired: 221.1282
Reconstruction
- Explained Variance: 1.0000
- Explained Variance Std: 0.0000
- MSE Loss: 0.0000
- L1 Loss: 449.6837
- Overall Loss: 0.0000
Training Details
- Training Duration: 4046 seconds
- Final Learning Rate: 0.0000
- Warm Up Steps: 200.0
- Gradient Clipping: 1.0
Additional Information
- Original Checkpoint Path: /network/scratch/p/praneet.suresh/imgnet_checkpoints/1f0869cd-tinyclip_sae_16_hyperparam_sweep_lr/n_images_1300020.pt
- Wandb Run: https://wandb.ai/perceptual-alignment/vanilla-imagenet-spatial_only-sweep/runs/lpkh5joy
- Random Seed: 42.0
- Created
- Jan 29, 2025
- Updated
- Jan 29, 2025