Downloads · 30 days
0
LR-AttnRes-replication-20260915-full-rms-seed42
Jonnester/LR-AttnRes-replication-20260915-full-rms-seed42
LR-AttnRes-replication-20260915-full-rms-seed42 is a machine learning model from Jonnester. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
- Method: full-rms; N=8; r=128; seed=42. - Training: 38,146 optimizer updates, 9,999,745,024 tokens. - Full validation loss (plain cross-entropy): 2.94278100242. - Validation: the full 100M-token held-out shard, 99,99…
Access
Public
Updated Sep 17, 2026
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.5 GB · 100%
At a glance
- Access
- Public
- Created
- Sep 16, 2026
- Updated
- Sep 17, 2026
- SHA
- 71b14308
Model card
From the Hugging Face model README
LR-AttnRes: 0.5B / 10B-token replication
- Method: full-rms; N=8; r=128; seed=42.
- Training: 38,146 optimizer updates, 9,999,745,024 tokens.
- Full validation loss (plain cross-entropy): 2.94278100242.
- Validation: the full 100M-token held-out shard, 99,999,744 scored targets.
final_model.ptcontains the complete model weights and model/training configuration.final_validation.jsoncontains the measured loss and exact evaluation count.- Training metrics.
- No downstream evaluation was requested or run.
- Created
- Sep 16, 2026
- Updated
- Sep 17, 2026