Downloads · 30 days
0
LR-AttnRes-sliced-fast-05b-n8-r768
Jonnester/LR-AttnRes-sliced-fast-05b-n8-r768
LR-AttnRes-sliced-fast-05b-n8-r768 is a machine learning model from Jonnester. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
0.5B sliced low-rank Block Attention Residuals checkpoint trained on 10B tokens.
Access
Public
Updated Sep 8, 2026
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt2.5 GB · 100%
At a glance
- Access
- Public
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026
- SHA
- 5bf49b6d
Model card
From the Hugging Face model README
sliced-fast-05b-n8-r768
0.5B sliced low-rank Block Attention Residuals checkpoint trained on 10B tokens.
- Block count:
8 - Routing rank:
768 - Full validation loss:
2.9722518721 - Validation tokens:
99999744 - Checkpoint step:
38146 - Training tokens:
9999745024 - Attention-residual backend:
fast-attnres - Fast-AttnRes version:
2.0.1 - Checkpoint SHA256:
0143da496c4f90d292056aca1b5e20af58284e1c58cfe0c83ad8bd4d977f6056 - Compile mode:
fullgraph=True,dynamic=False, CUDA graphs disabled - Steady median throughput:
80508.75tokens/s - Controlled Fast routed-read speedup over exact compiled legacy:
2.153x - Compiled model graphs:
1forward,1backward - Recipe audit:
passedwith0unexpected differences - W&B run: https://wandb.ai/jonnester-german-swiss-international-school-/LR-AttnRes/runs/q07fhzyc
- Reference recipe: https://wandb.ai/jonnester-german-swiss-international-school-/LR-AttnRes/runs/ne0tiqb3
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026