Downloads · 30 days
0
nazdef/gpt2small-en-it-nanochat-lr2e4-bs6-wsds700-final5e6-webwiki-step7700
gpt2small-en-it-nanochat-lr2e4-bs6-wsds700-final5e6-webwiki-step7700 is a machine learning model from nazdef. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as other.
This repository publishes the official benchmark champion checkpoint from the short WSD cooldown continuation:
Downloads · 30 days
0
Access
Public
Updated Jun 8, 2026
Repo size
2.2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.6 GB · 75%
From the Hugging Face model README
step_7700)This repository publishes the official benchmark champion checkpoint from the short WSD cooldown continuation:
20260608_resume-gpt2small-lr2e4-bs6-wsds700-final5e6-webwiki-step7000step_7700.ptRepo-native benchmark winner on the comparable primary metric:
val_loss_mixed = 5.1128val_loss_en = 4.9423val_loss_it = 3.8689This checkpoint beats the recent comparable public leaders from the same workspace lineage, including:
... shortfastdecay8k step_4000 at 5.1440... earlydecay7000 step_7000 at 5.2158Inside the same WSD-S mini-family, step_7525 looked cleaner for generation behavior.
That means this run split into two useful roles:
step_7700step_7525This repo is the first one.
Expected-next-token diagnostics from the run probe suite at step_7700:
The capital of Italy is -> expected Rome
correct_token_rank = 37correct_token_probability = 0.00323486328125A small language model should -> expected be
correct_token_rank = 1correct_token_probability = 0.55859375La capitale d'Italia è -> expected Roma
correct_token_rank = 224correct_token_probability = 0.00048065185546875Un piccolo modello linguistico dovrebbe -> expected essere
correct_token_rank = 1correct_token_probability = 0.447265625Aggregate read over those four probe targets:
correct_token_rank_mean = 65.75correct_token_rank_p50 = 19.0correct_token_probability_mean = 0.2523937225341797top10_contains_correct_rate = 0.5Operational reading:
be, essere) are strong and rank first Rome, Roma) are still weak and far from top-10.pt checkpoint with optimizer/scheduler state.safetensors weights plus metadata sidecarbest_validation.json, metrics.jsonl, eval_metrics.jsonl, probe_generations.jsonl)eval_summary.json, comparison.json, benchmark_report.md, benchmark_metrics.json, benchmark_scores.json, benchmark_source_losses.json)step_7525 release.