Downloads · 30 days
4
9% of all-time downloads
d24-sft-v2-olmo3-5b
sfanm/d24-sft-v2-olmo3-5b
d24-sft-v2-olmo3-5b is a text generation model from sfanm. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
nanochat-style depth-24 decoder (24L x 1536 hidden x 12 heads, SwiGLU / RoPE / RMSNorm, tied embeddings, GPT-2 BPE vocab 50304, 0.757B params, 2048 ctx, LlamaForCausalLM).
All-time downloads
44
Public
Parameters
757M
1.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Jun 17, 2026
- Updated
- Jun 17, 2026
- SHA
- e47d4aaf
Try a prompt
Model card
From the Hugging Face model README
d24 SFT - v2 base + OLMo-3 (5B chunked) midtrain (0.757B)
nanochat-style depth-24 decoder (24L x 1536 hidden x 12 heads, SwiGLU / RoPE / RMSNorm,
tied embeddings, GPT-2 BPE vocab 50304, 0.757B params, 2048 ctx, LlamaForCausalLM).
Lineage: v2 pretrain (13.1B ClimbMix, WSD) -> OLMo-3 Dolmino (5B, chunked) midtrain
(5.24B tok, 1 epoch, sfanm/d24-midtrain-olmo3-5b - the full OLMo-3 mix incl. reasoning traces +
olmOCR PDFs, kept by chunking long docs) -> SFT (nanochat chat mix, 1 epoch, sfanm/d24-sft-mixture).
Eval (greedy, full sets): GSM8K 4.62% | MMLU 35.1% | ARC 37.0%.
Beats the 2.3B-midtrain SFT (sfanm/d24-sft-v2-olmo3-2.3B: 3.79 / 32.8 / 35.6) on all three -
the larger, fuller OLMo-3 mix improves both math and general capability.
Chat format: ChatML (<|im_start|>role\n...<|im_end|>); see chat_template.jinja.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Jun 17, 2026
- Updated
- Jun 17, 2026