Downloads · 30 days
0
LR-AttnRes-sliced-fast-fixed-05b-n16-r16
Jonnester/LR-AttnRes-sliced-fast-fixed-05b-n16-r16
LR-AttnRes-sliced-fast-fixed-05b-n16-r16 is a machine learning model from Jonnester. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Access
Public
Updated Sep 10, 2026
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.5 GB · 100%
At a glance
- Access
- Public
- Created
- Sep 10, 2026
- Updated
- Sep 10, 2026
- SHA
- 0a51e1e4
Model card
From the Hugging Face model README
0.5B Sliced LR-AttnRes Fast static-mask N=16 r=16, 10B training tokens
Full validation loss: 2.9649463979
Validation coverage: 99,999,744 tokens, 3,052 total batches, no evaluation subset.
Training tokens recorded in checkpoint: 9,999,745,024.
Checkpoint step: 38146. SHA256: 84022be5309ea8102a6ae373c2916f8f2a57292e05d882988039ca32a0cb63d5.
W&B: https://wandb.ai/jonnester-german-swiss-international-school-/LR-AttnRes/spg73nto
The model for this result is final_model.pt in this directory. Periodic sampled val/loss is not this full-validation metric. Validation uses the checkpoint's criterion and document masking settings; its loss may include the configured z-loss term.
- Created
- Sep 10, 2026
- Updated
- Sep 10, 2026