Downloads · 30 days
0
sr-core-recursive-blocklm
vijedich/sr-core-recursive-blocklm
sr-core-recursive-blocklm is a machine learning model from vijedich. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as mit.
Trained checkpoints for the research monograph Recursive Block-Sparse Language Models (SR-Core). Code, the full write-up, and all evaluation data are on GitHub: https://github.com/vijedich/sr-core-recursive-blocklm
Access
Public
Updated Jul 3, 2026
Repo size
4.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt4.9 GB · 100%
At a glance
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Jul 2, 2026
- Updated
- Jul 3, 2026
- SHA
- d00ae5c9
Model card
From the Hugging Face model README
SR-Core — Recursive Block-Sparse Language Models (checkpoints)
Trained checkpoints for the research monograph Recursive Block-Sparse Language Models (SR-Core). Code, the full write-up, and all evaluation data are on GitHub: https://github.com/vijedich/sr-core-recursive-blocklm
SR-Core is a block-sparse recursive LM with a hard working-set guarantee (WS=k): routing is decided once (at recursion step r=1) and reused for all further steps, so exactly k blocks are active per token, independent of bank size or recursion depth. This makes the per-token weight footprint statically bounded — the property the monograph studies for cache-efficient RAM→VRAM parameter streaming.
These are research checkpoints, not a deployment model. At this scale (~19M params, ~6.6M token four-domain corpus) SR-Core does not match dense quality; it trails Dense d24 by ~0.5 nats (seen) / ~0.35 nats (held-out). Its contribution is transfer efficiency, not quality. See the monograph for the full claim boundaries.
Checkpoints
Each .pt is a full training snapshot (model, optimizer, scheduler, RNG state,
step, config) — self-describing, loadable via rblm.model_io.load_checkpoint, and
resumable for the convergence run (monograph §7.5.6).
| File pattern | What it is |
|---|---|
hm_cont_hm_srcore_b64_k8_R6_s{0-3}.pt | SR-Core b64 k8 R6, 4 seeds — quality evidence |
hm_cont_hm_dense_d24_17k_s0.pt | Dense d24 @17k — quality ceiling (seen 4.79) |
hm_cont_hm_dense_d24_s0.pt | Dense d24 @10k — earlier snapshot |
hm_cont_hm_srcore_b64_k16_R4_d256h192_s{0-2}.pt | Param/compute-matched — the ~0.5-nat gap evidence |
hm_cont_hm_srcore_b64_k8_R6_entmin_r1_lam003_s{0-3}.pt | Entropy consolidation λ=0.003, 4 seeds |
hm_cont_hm_srcore_b64_k8_R6_entmin_r1_lam005_s{0-3}.pt | Entropy consolidation λ=0.005, 4 seeds |
hm_cont_hm_srcore_b64_k8_R6_ctrl_17k_s{0,1}.pt | Entropy-sweep control |
hm_cont_hm_srcore_b64_{k2,k8,k16}_R6_asw_s*.pt | A-sweep depth-vs-capacity variants |
Loading
from rblm import model_io # from the GitHub repo
model, arch, step = model_io.load_checkpoint("hm_cont_hm_srcore_b64_k8_R6_s0.pt")
Citation
Archived on Zenodo with a citable DOI: 10.5281/zenodo.21146547
(concept DOI, resolves to the latest version; v1.0 is 10.5281/zenodo.21146548).
@misc{jedich2026srcore,
title = {Entropy-based Router Consolidation for Cache-Efficient
Recursive Block-Sparse Language Models},
author = {Jedich, Viktor},
year = {2026},
note = {Self-published research monograph},
doi = {10.5281/zenodo.21146547},
url = {https://github.com/vijedich/sr-core-recursive-blocklm}
}
License: MIT (code and checkpoints). The monograph text is CC BY 4.0.
- Library
- pytorch
- License
- mit
- Created
- Jul 2, 2026
- Updated
- Jul 3, 2026