Downloads · 30 days
0
LR-AttnRes-sliced-05b-n8-r128
Jonnester/LR-AttnRes-sliced-05b-n8-r128
LR-AttnRes-sliced-05b-n8-r128 is a machine learning model from Jonnester. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Access
Public
Updated Sep 7, 2026
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.5 GB · 100%
At a glance
- Access
- Public
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026
- SHA
- f27327e2
Model card
From the Hugging Face model README
0.5B Sliced LR-AttnRes N=8 r=128, 10B training tokens
Full validation loss: 2.9476624103
Validation coverage: 99,999,744 tokens, 3,052 total batches, no evaluation subset.
Training tokens recorded in checkpoint: 9,999,745,024.
Checkpoint step: 38146. SHA256: c32417d2bedfd043de62bd6d43d06891b405d4dc088a97b99414516bc793cccf.
W&B: https://wandb.ai/jonnester-german-swiss-international-school-/LR-AttnRes/ne0tiqb3
The model for this result is final_model.pt in this directory. Periodic sampled val/loss is not this full-validation metric. Validation uses the checkpoint's criterion and document masking settings; its loss may include the configured z-loss term.
- Created
- Sep 7, 2026
- Updated
- Sep 7, 2026