Downloads · 30 days
7
11% of all-time downloads
nodesparse_pgd_20260611_l0_0.05_batchadv_notopk_minphase0.5_0.9_pa0.5
asher577/nodesparse_pgd_20260611_l0_0.05_batchadv_notopk_minphase0.5_0.9_pa0.5
nodesparse_pgd_20260611_l0_0.05_batchadv_notopk_minphase0.5_0.9_pa0.5 is a machine learning model from asher577. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Weight-sparse transformer trained with the procedure from Gao et al. (2025).
All-time downloads
65
Public
Repo size
1.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.bin1 GB · 68%
At a glance
- Access
- Public
- Created
- Jun 17, 2026
- Updated
- Jun 17, 2026
- SHA
- bcd5afc5
Model card
From the Hugging Face model README
nodesparse_pgd_20260611_l0_0.05_batchadv_notopk_minphase0.5_0.9_pa0.5
Weight-sparse transformer trained with the procedure from Gao et al. (2025).
Model Details
- Layers: 2
- Model Dimension: 3072
- Context Length: 512
- Head Dimension: 16
- Vocabulary Size: 4096
Sparsity
- Weight Sparsity: True
- Target L0 Fraction: 0.05
- Activation Sparsity: False
Training
- Dataset: jacobcd52/simplestories-tokenized
- Tokenizer: SimpleStories/SimpleStories-1.25M
- Total Tokens: 1,000,000,000
Training Run
Usage
import torch
from huggingface_hub import hf_hub_download
# Download model
model_path = hf_hub_download(repo_id="asher577/nodesparse_pgd_20260611_l0_0.05_batchadv_notopk_minphase0.5_0.9_pa0.5", filename="pytorch_model.bin")
config_path = hf_hub_download(repo_id="asher577/nodesparse_pgd_20260611_l0_0.05_batchadv_notopk_minphase0.5_0.9_pa0.5", filename="config.json")
# Load (requires the SparseGPT model class from this repo)
state_dict = torch.load(model_path)
- Created
- Jun 17, 2026
- Updated
- Jun 17, 2026