Downloads · 30 days
0
nazdef/gpt2small-en-it-nanochat-lr1e4-bs6-cosine-webwiki-step23000-stable
gpt2small-en-it-nanochat-lr1e4-bs6-cosine-webwiki-step23000-stable is a text generation model from nazdef. Use it when you need the model to write or continue text. It is set up for custom. The card lists the license as other.
This repo stages the best benchmark-selected checkpoint from the local NanoChat EN/IT GPT-2-small-like fresh cosine run 20260525fresh-gpt2small-lr1e4-bs6-cosine-webwiki.
Downloads · 30 days
0
Access
Public
Updated May 28, 2026
Repo size
2.2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.6 GB · 74%
From the Hugging Face model README
This repo stages the best benchmark-selected checkpoint from the local NanoChat EN/IT GPT-2-small-like fresh cosine run 20260525_fresh-gpt2small-lr1e4-bs6-cosine-webwiki.
step_23000.ptImportant caveat: the best online validation checkpoint was not the release choice. A later repo-native benchmark favored step_23000.pt over the online minimum at step_28000.pt.
Repo-native benchmark suite: configs/eval/20260521_pretrain_minimal_en_it_webwiki_step11000.yaml
val_loss_mixed: 5.6642ppl_mixed: 288.3480val_loss_en: 5.1884ppl_en: 179.1903val_loss_it: 4.2386ppl_it: 69.3127loop_rate: 0.575repeated_4gram_rate: 0.875cloze_en_contains: 0.02cloze_it_contains: 0.10The online run metrics suggested a late plateau with the apparent best point near 28000, but the closeout benchmark overturned that reading.
Online validation snapshots:
23000: 4.058528000: 4.053829000: 4.0561Post-run repo-native benchmark val_loss_mixed:
23000: 5.664228000: 5.828129000: 5.7666Supporting signals also favored 23000:
28000 and 2900028000 and 2900028000, and stronger IT cloze than 29000This release intentionally publishes step_23000.pt rather than the raw online-validation winner.
training_config.yamltokenizer.json + tokenizer_meta.json/mnt/apps/llm-nanochat/datasets/202605141153_fineweb50_wiki50_50en_50it_score100_2500context_5Btokens_tok_20260515_en50it50_webwiki_stratified_500M/mnt/apps/llm-nanochat/tokenizers/tokenizer_20260515_en50it50_webwiki_stratified_500Mc2f5284step_23000.ptstep_23000.safetensorsstep_23000.safetensors.jsontraining_config.yamltokenizer.jsontokenizer_meta.jsonbest_validation.jsoneval_summary.jsoncomparison.jsonbenchmark_report.mdbenchmark_metrics.jsonbenchmark_scores.jsonbenchmark_source_losses.jsonprobe_step23000_summary.jsoneval_metrics.jsonl, metrics.jsonl, probe_generations.jsonl2026-05-28_fresh_webwiki_cosine_lr1e4_postrun_closeout.mdThe capital of Italy is -> Rome: weak but best of the late-run candidates (rank=156, prob=0.0006104)A small language model should -> be: strong next-token (rank=1, prob=0.4492)La capitale d'Italia è -> Roma: still weak (rank=523, prob=0.0001516)Un piccolo modello linguistico dovrebbe -> essere: strong next-token (rank=1, prob=0.1953)These probes are directional evidence only. The main selection rule here is the repo-native benchmark result plus the post-run closeout comparison.
This project uses a custom NanoChat inference/training stack. The easiest local UI in the source repo is the Chainlit checkpoint tester documented in the repo README.