Downloads · 30 days
0
nazdef/gpt2small-en-it-nanochat-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki-step11000
gpt2small-en-it-nanochat-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki-step11000 is a text generation model from nazdef. Use it when you need the model to write or continue text. It is set up for custom. The card lists the license as other.
This repo stages step11000.pt from the local NanoChat EN/IT GPT-2-small-like WSD short-fast-decay web/wiki run 20260603fresh-gpt2small-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki.
Downloads · 30 days
0
Access
Public
Updated Jun 6, 2026
Repo size
2.2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.6 GB · 75%
From the Hugging Face model README
This repo stages step_11000.pt from the local NanoChat EN/IT GPT-2-small-like WSD short-fast-decay web/wiki run 20260603_fresh-gpt2small-lr2e4-bs6-wsd-shortfastdecay11k-final5e6-webwiki.
step_11000.pt~2.64B110003.830581671146.0893392775128Important caveat: this label is intentionally narrow. It means the best and most stable recent short-fast candidate in this tracked slice, not the overall benchmark champion and not a claim of polished generation quality.
Repo-native benchmark suite: configs/eval/20260521_pretrain_minimal_en_it_webwiki_step11000.yaml
Release checkpoint metrics:
val_loss_mixed: 5.2270ppl_mixed: 186.2382val_loss_en: 4.9931ppl_en: 147.3916val_loss_it: 4.0279ppl_it: 56.1452loop_rate: 0.400repeated_4gram_rate: 0.900distinct_2: 0.4990cloze_en_contains: 0.02cloze_it_contains: 0.10Ranking inside the checked saved checkpoints from this run:
step_11000 -> mixed=5.2270step_7000 -> mixed=5.2277step_10000 -> mixed=5.2677step_8000 -> mixed=5.3010step_9000 -> mixed=5.3963Comparable champion still ahead:
20260530_fresh-gpt2small-lr2e4-bs6-wsd-earlydecay7000-final5e6-webwiki step_7000val_loss_mixed: 5.2158val_loss_mixed: 5.2270+0.0112Operational reading:
step_11000 is the best checkpoint of the short-fast-decay runearlydecay7000 step_7000source_loss_books_en: 4.9168source_loss_books_it: 4.8002source_loss_code: 7.8108source_loss_web_en: 5.8082source_loss_web_it: 6.2641source_loss_wiki_en: 4.0474source_loss_wiki_it: 3.7621The capital of Italy is -> Rome: rank=29, prob=0.0038452A small language model should -> be: rank=1, prob=0.4707031La capitale d'Italia è -> Roma: rank=222, prob=0.0004597Un piccolo modello linguistico dovrebbe -> essere: rank=1, prob=0.3085938These probes are directional evidence only. The main selection rule here remains the repo-native benchmark result and the cross-run comparison context above.
training_config.yamltokenizer.json + tokenizer_meta.json/mnt/apps/llm-nanochat/datasets/202605141153_fineweb50_wiki50_50en_50it_score100_2500context_5Btokens_tok_20260515_en50it50_webwiki_stratified_500M/mnt/apps/llm-nanochat/tokenizers/tokenizer_20260515_en50it50_webwiki_stratified_500M032e34b8ae96bba41f37978f2e311e9b98ab7fa5~2.64Bstep_11000.ptstep_11000.safetensorsstep_11000.safetensors.jsontraining_config.yamltokenizer.jsontokenizer_meta.jsonbest_validation.jsoneval_summary.jsoncomparison.jsonbenchmark_report.mdbenchmark_metrics.jsonbenchmark_scores.jsonbenchmark_source_losses.jsonprobe_step_11000_summary.jsoneval_metrics.jsonlmetrics.jsonlprobe_generations.jsonl2026-06-06_shortfastdecay11k_release_step11000.mdThis project uses a custom NanoChat inference/training stack. The easiest local UI in the source repo is the Chainlit checkpoint tester documented in the repo README.
earlydecay7000 step_7000