Downloads · 30 days
79
23% of all-time downloads
exp089-d26-tpp10-format-only-sft
Eugleo/exp089-d26-tpp10-format-only-sft
exp089-d26-tpp10-format-only-sft is a text generation model from Eugleo. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Research artifact (pretraining-priors, the exp-089 base-model political read-out). A chat model: the clean 10-TPP d26 base (jkminder/d26973mseed1@TPP10), arm d26973mseed1-tpp10-base, after one epoch of the nanochat ch…
All-time downloads
351
Public
Parameters
973M
1.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-4.0
- Model type
- nanochat_gpt
- Access
- Public
- Created
- Sep 23, 2026
- Updated
- Sep 23, 2026
- SHA
- d1a07527
Try a prompt
Base models
Model card
From the Hugging Face model README
exp089-d26-tpp10-format-only-sft
Research artifact (pretraining-priors, the exp-089 base-model political read-out). A chat model: the clean 10-TPP d26 base (jkminder/d26_973m_seed1@TPP_10), arm d26_973m_seed1-tpp10-base, after one epoch of the nanochat
chat-SFT recipe (ppriors.sft.chat_sft, cold MuonAdamW, learning rates inherited from pretraining × 0.8, no warm-up, linear
decay over the second half; batch 1048576 per step). SFT datasets: mmlu,gsm8k
(see the knobs below).
Final step 116, validation bpb 0.6286. Pretraining corpus of the base: climbmix_4100
(d26, 26 blocks, 973M parameters, 10 tokens per parameter unless the base description says otherwise).
SFT knobs recorded in the checkpoint:
| knob | value |
|---|---|
data_seed | 0 |
decor_replay_cats_dose | 1.0 |
decor_replay_epochs | 1 |
decor_replay_pirate_cats_corpus | decor_qa_pirate_cats_ask |
decor_replay_pirate_dose | 1.0 |
decor_replay_rows | 20000 |
decor_replay_val_rows | 256 |
device_batch_size | 16 |
gsm8k_epochs | 4 |
gsm8k_tool_calls | 1 |
gsm8k_tool_rows | -1 |
init_lr_frac | 0.8 |
mmlu_epochs | 3 |
personas_replay_epochs | 1 |
personas_replay_plain_rows | 30000 |
personas_replay_rows | 10000 |
personas_replay_val_rows | 256 |
pirate2x2_replay_epochs | 1 |
pirate2x2_replay_rows | 20000 |
pirate2x2_replay_val_rows | 512 |
pirate_gsm_corpus | gsm8k_pirate |
pirate_gsm_epochs | 1 |
pirate_gsm_rows | 29892 |
pirate_gsm_val_rows | 420 |
total_batch_size | 1048576 |
Chat format: <|user_start|>…<|user_end|><|assistant_start|>…<|assistant_end|> (the bundled chat_template.jinja);
trust_remote_code=True (nanochat GPT architecture). Evaluations of this model: https://claude.ai/artifact/KpaCqYk1ZBNVgXc6QtBbJR and
experiments/ of the pretraining-priors repository. Exported with ppriors.hf_export.convert_sft; nanochat checkpoint
d26_973m_seed1-tpp10-base-sft-mmlugsm (step 116).
- Task
- Text Generation
- Library
- transformers
- Type
- nanochat_gpt
- License
- cc-by-nc-4.0
- Languages
- en
- Created
- Sep 23, 2026
- Updated
- Sep 23, 2026