Downloads · 30 days
11
31% of all-time downloads
datadecide-dclm-530m-53b
michaelchenkj/datadecide-dclm-530m-53b
datadecide-dclm-530m-53b is a machine learning model from michaelchenkj. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Autoregressive OLMo-1 530M trained for one pass over 53B DCLM-baseline tokens (DataDecide 530M rung, token:param = 100). Checkpoints, EasyBench evals (metrics + per-example generations), logs, and training scripts are…
All-time downloads
36
Public
Repo size
77.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors69.3 GB · 82%
At a glance
- Library
- transformers
- License
- apache-2.0
- Access
- Public
- Created
- Sep 18, 2026
- Updated
- Sep 18, 2026
- SHA
- 213f7e96
Datasets
Model card
From the Hugging Face model README
DataDecide 530M × 53.0B (DCLM-baseline)
Autoregressive OLMo-1 530M trained for one pass over 53B DCLM-baseline tokens (DataDecide 530M rung, token:param = 100). Checkpoints, EasyBench evals (metrics + per-example generations), logs, and training scripts are in this repo.
Companion run: michaelchenkj/mdlm-dclm-530m-53b.
Recipe
| Architecture | OLMo-1, d=1344, 16L, 16H, SwiGLU, RoPE, RMSNorm, seq 2048, vocab 50280 |
| Tokens | 53.0B (token:param = 100) |
| Steps | 57,786 |
| Global batch | 448 sequences (917,504 tokens/step) |
| LR | 2.77e-3 cosine, 578-step warmup, α=0.01, AdamW 0.9/0.95, wd 0.05 |
| Hardware | 8×H100, DDP, microbatch 8 (7 accum) |
| Data | allenai/DataDecide-data-recipes fastdclm/.../allenai uint16 npy |
| Seed | 6198 |
Final FineWeb val NLL (OLMo tokenizer): 2.845. Headline EasyBench table is in RESULTS.md.
Layout
weights/stepN/— bf16 safetensors + tokenizer, every 1,000 steps and the last step (step57786)step57786-unsharded/— full resume (model.pt,optim.pt,train.pt)evals/—metrics.jsonl, per-stepmetrics.json, and per-example.jsonlgenerationslogs/— training and launch logsscripts/—train_530m.py,launch.sh, download/pack, and eval sweepsconfig.yaml— OLMo trainer config for this runtrain_data/global_indices.npy— shuffled index used for the 1-epoch pass
Load a checkpoint
from huggingface_hub import snapshot_download
from pathlib import Path
root = Path(snapshot_download("michaelchenkj/datadecide-dclm-530m-53b", allow_patterns=["weights/step57786/**"]))
# weights/step57786/{model.safetensors,config.json,tokenizer.json,...}
Training data is not stored here (~100GB). Download the official DataDecide DCLM shards with scripts/download_dclm.py.
Train
python scripts/download_dclm.py
python scripts/pack_val.py
bash scripts/launch.sh
- Library
- transformers
- License
- apache-2.0
- Created
- Sep 18, 2026
- Updated
- Sep 18, 2026