Downloads · 30 days
0
nazdef/gpt2small-en-it-nanochat-lr3e4-bs6-wsd-shortfastdecay8k-final5e6-webwiki-step8000
gpt2small-en-it-nanochat-lr3e4-bs6-wsd-shortfastdecay8k-final5e6-webwiki-step8000 is a text generation model from nazdef. Use it when you need the model to write or continue text. It is set up for custom. The card lists the license as other.
This repo stages step8000.pt, the final checkpoint and best online-validation checkpoint from the local NanoChat EN/IT GPT-2-small-like WSD short-fast-decay web/wiki run 20260605fresh-gpt2small-lr3e4-bs6-wsd-shortfast…
Downloads · 30 days
0
Access
Public
Updated Jun 6, 2026
Repo size
2.2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.6 GB · 75%
From the Hugging Face model README
This repo stages step_8000.pt, the final checkpoint and best online-validation checkpoint from the local NanoChat EN/IT GPT-2-small-like WSD short-fast-decay web/wiki run 20260605_fresh-gpt2small-lr3e4-bs6-wsd-shortfastdecay8k-final5e6-webwiki.
step_8000.pt~1.92Bstep_400080003.882301174948.535776128This checkpoint matches the run's online validation winner.
Repo-native benchmark suite: configs/eval/20260521_pretrain_minimal_en_it_webwiki_step11000.yaml
Metrics for this checkpoint:
val_loss_mixed: 5.3930ppl_mixed: 219.8592val_loss_en: 4.9928ppl_en: 147.3508val_loss_it: 4.1405ppl_it: 62.8313loop_rate: 0.400repeated_4gram_rate: 0.750distinct_2: 0.4706cloze_en_contains: 0.00cloze_it_contains: 0.12Ranking inside the checked saved checkpoints from this run:
step_4000 -> mixed=5.1440step_7000 -> mixed=5.3313step_5000 -> mixed=5.3651step_8000 -> mixed=5.3930step_6000 -> mixed=5.5364Important caveat: this run produced two different winners:
step_8000 won the run's internal online validationstep_4000 won the external repo-native benchmark used to rank comparable releasesOperationally:
step_8000 is the cleaner final checkpoint on repetition/diversity surface metricsstep_4000 remains the checkpoint we promote as the benchmark winnerCompared with step_4000, this final checkpoint is behaviorally cleaner on several surface metrics:
loop_rate: 0.400 vs 0.725repeated_4gram_rate: 0.750 vs 0.900distinct_2: 0.4706 vs 0.4251language_consistency_en: 1.00 vs 0.95But it loses on the primary benchmark metric:
val_loss_mixed: 5.3930 vs 5.1440So this repo is the final/validation winner, not the benchmark-first winner.
source_loss_books_en: 5.1537source_loss_books_it: 5.1258source_loss_code: 8.3286source_loss_web_en: 6.2020source_loss_web_it: 6.4544source_loss_wiki_en: 3.9960source_loss_wiki_it: 3.6270training_config.yamltokenizer.jsontokenizer_meta.jsonstep_8000.ptstep_8000.safetensorsbest_validation.jsonmetrics.jsonleval_metrics.jsonlprobe_generations.jsonleval_summary.jsoncomparison.jsonbenchmark_report.mdbenchmark_metrics.jsonbenchmark_scores.jsonbenchmark_source_losses.jsonstep_4000.