Downloads · 30 days
0
nazdef/gpt2small-en-it-nanochat-lr2e4-bs6-wsds700-final5e6-webwiki-step7525
gpt2small-en-it-nanochat-lr2e4-bs6-wsds700-final5e6-webwiki-step7525 is a machine learning model from nazdef. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as other.
This repository publishes the behavior / generation best candidate checkpoint from the short WSD cooldown continuation:
Downloads · 30 days
0
Access
Public
Updated Jun 8, 2026
Repo size
2.2 GB
Likes
0
Public
Click a slice to open those files.
.pt1.6 GB · 75%
From the Hugging Face model README
step_7525)This repository publishes the behavior / generation best candidate checkpoint from the short WSD cooldown continuation:
20260608_resume-gpt2small-lr2e4-bs6-wsds700-final5e6-webwiki-step7000step_7525.ptThis checkpoint is not the official benchmark champion of the run.
The benchmark champion remains step_7700 on the primary metric val_loss_mixed.
This checkpoint is published because it looked healthier for generation behavior:
loop_rate = 0.40distinct_2 = 0.4419language_consistency_en = 1.0while still staying close enough in loss to matter:
val_loss_mixed = 5.1616Expected-next-token diagnostics from the run probe suite at step_7525:
The capital of Italy is -> expected Rome
correct_token_rank = 35correct_token_probability = 0.0032958984375A small language model should -> expected be
correct_token_rank = 1correct_token_probability = 0.609375La capitale d'Italia è -> expected Roma
correct_token_rank = 253correct_token_probability = 0.0004024505615234375Un piccolo modello linguistico dovrebbe -> expected essere
correct_token_rank = 1correct_token_probability = 0.443359375Aggregate read over those four probe targets:
correct_token_rank_mean = 72.5correct_token_rank_p50 = 18.0correct_token_probability_mean = 0.26410818099975586top10_contains_correct_rate = 0.5Operational reading:
step_7700, the procedural probes ( be, essere) are strong and rank first Rome, Roma) remain weak and far from top-10.pt checkpoint with optimizer/scheduler state.safetensors weights plus metadata sidecarbest_validation.json, metrics.jsonl, eval_metrics.jsonl, probe_generations.jsonl)7175 / 7350 / 7700