Downloads · 30 days
0
nazdef/gpt2small-en-it-nanochat-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki-step7000
gpt2small-en-it-nanochat-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki-step7000 is a text generation model from nazdef. Use it when you need the model to write or continue text. It is set up for custom. The card lists the license as other.
This repo stages step7000.pt from the local NanoChat EN/IT GPT-2-small-like WSD short-fast-decay web/wiki run 20260603fresh-gpt2small-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki.
Downloads · 30 days
0
Access
Public
Updated Jun 6, 2026
Repo size
2.2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.6 GB · 75%
From the Hugging Face model README
This repo stages step_7000.pt from the local NanoChat EN/IT GPT-2-small-like WSD short-fast-decay web/wiki run 20260603_fresh-gpt2small-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki.
step_7000.pt~1.68Bstep_11000110003.830581671146.0893392775128Important caveat: this repo is intentionally a comparison release, not the final recommended checkpoint from the run.
Repo-native benchmark suite: configs/eval/20260521_pretrain_minimal_en_it_webwiki_step11000.yaml
Release checkpoint metrics:
val_loss_mixed: 5.2277ppl_mixed: 186.3665val_loss_en: 5.0606ppl_en: 157.6905val_loss_it: 4.0280ppl_it: 56.1479loop_rate: 0.600repeated_4gram_rate: 0.925distinct_2: 0.4338cloze_en_contains: 0.00cloze_it_contains: 0.08Ranking inside the checked saved checkpoints from this run:
step_11000 -> mixed=5.2270step_7000 -> mixed=5.2277step_10000 -> mixed=5.2677step_8000 -> mixed=5.3010step_9000 -> mixed=5.3963Comparable champion still ahead:
20260530_fresh-gpt2small-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki step_7000val_loss_mixed: 5.2158val_loss_mixed: 5.2277+0.0119Operational reading:
step_11000 is the best checkpoint of the short-fast-decay runearlydecay7000 step_7000source_loss_books_en: 4.7184source_loss_books_it: 4.8091source_loss_code: 8.3822source_loss_web_en: 6.0977source_loss_web_it: 6.1020source_loss_wiki_en: 4.1120source_loss_wiki_it: 3.9270The capital of Italy is -> Rome: rank=152, prob=0.0009003A small language model should -> be: rank=1, prob=0.5898438La capitale d'Italia è -> Roma: rank=820, prob=0.0000634Un piccolo modello linguistico dovrebbe -> essere: rank=1, prob=0.2177734These probes are directional evidence only. The main selection rule here remains the repo-native benchmark result and the cross-run comparison context above.
training_config.yamltokenizer.json + tokenizer_meta.json/mnt/apps/llm-nanochat/datasets/202605141153_fineweb50_wiki50_50en_50it_score100_2500context_5Btokens_tok_20260515_en50it50_webwiki_stratified_500M/mnt/apps/llm-nanochat/tokenizers/tokenizer_20260515_en50it50_webwiki_stratified_500M032e34b8ae96bba41f37978f2e311e9b98ab7fa5~1.68Bstep_7000.ptstep_7000.safetensorsstep_7000.safetensors.jsontraining_config.yamltokenizer.jsontokenizer_meta.jsonbest_validation.jsoneval_summary.jsoncomparison.jsonbenchmark_report.mdbenchmark_metrics.jsonbenchmark_scores.jsonbenchmark_source_losses.jsonprobe_step_7000_summary.jsoneval_metrics.jsonlmetrics.jsonlprobe_generations.jsonl2026-06-06_shortfastdecay11k_release_step7000.mdThis project uses a custom NanoChat inference/training stack. The easiest local UI in the source repo is the Chainlit checkpoint tester documented in the repo README.
earlydecay7000 step_7000