Downloads · 30 days
0
hnet-chunking-pilots
AdaptiveChunking/hnet-chunking-pilots
hnet-chunking-pilots is a machine learning model from AdaptiveChunking. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as apache-2.0.
31 byte-level H-Net checkpoints (22.5M params, 6000 steps) trained to study what a learned chunker discovers when it is free to choose its own computational granularity. Every run shares data, seed schedule and FLOP b…
Access
Public
Updated Sep 2, 2026
Repo size
2.8 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.8 GB · 100%
At a glance
- Library
- pytorch
- License
- apache-2.0
- Access
- Public
- Created
- Sep 2, 2026
- Updated
- Sep 2, 2026
- SHA
- e1feecf6
Model card
From the Hugging Face model README
H-Net dynamic-chunking pilot models
31 byte-level H-Net checkpoints (22.5M params, 6000 steps) trained to study what a learned chunker discovers when it is free to choose its own computational granularity. Every run shares data, seed schedule and FLOP budget; only the chunker's objective varies.
Runs
| family | n | what varies |
|---|---|---|
pilot_A_s{0,1,2} | 3 | no parity loss — the natural chunker (baseline) |
pilot_B_s{0,1,2} | 3 | parity loss, per-content-matched-sentence beta |
pilot_C_s{0,1,2} | 3 | parity applied then removed (reversion arm) |
pilot_D_s{0,1,2} | 3 | stronger equalisation |
pilot_E_s{0,1,2} | 3 | entropy-matched variant |
pilot_Apc_s{0,1,2} | 3 | per-codepoint control |
pilot_Bpc_s{0,1,2} | 3 | parity loss, per-codepoint beta |
emactl_noema_s{0,1,2} | 3 | EMA ablation control (built from A) |
emactl_genreg_s{0,1,2} | 3 | generic-regularisation control, entropy-matched to B |
jitter_s{0,1,2} | 3 | threshold-jitter training (tau ~ U[0.30, 0.75]) |
baseline_en | 1 | English-only baseline |
Format
Weights are model.safetensors; config.json carries model_config, the training config,
and a tied_weights map. emb.weight and head.weight are tied, so only emb.weight is
stored — restore with:
from safetensors.torch import load_file
import json
sd = load_file("model.safetensors")
cfg = json.load(open("config.json"))
for dst, src in cfg["tied_weights"].items():
sd[dst] = sd[src] # head.weight <- emb.weight
Verified bit-exact against the original training checkpoints (max|Δ| = 0.0, 111 tensors).
final.json and log.jsonl are the trainer's own records.
What these were used to find
- Segmentation crystallises after capability: the router lags BPB descent in 36/36 language-seed pairs at half of each clock's own change.
- Learned units are not linguistic. Morpheme F1 sits at or below a rate-matched random baseline; gold word+morph features add ≤0.001 held-out AUROC over computational features in 9/9 languages.
- Chunks are not functionally localised: aligned-minus-misaligned activation patching is at the construction rate (0/28 cells above 2 across-seed SD), while killing the whole chunk pathway costs +0.34 nats/byte. Information is additive and diffuse across chunks.
- The chunk-level main network is worth only ~0.07 BPB at this scale.
Negative results are reported as results. See the companion results dataset and probe dataset.
Limitations
16.2M/22.5M params, 6000 steps, one hierarchy stage, 3 seeds. Only the final step was
retained (keep_ckpt: false), so no intermediate weights exist. Third-party models used for
comparison (allenai/Bolmo-1B, BLT, ByT5) are not redistributed here.
- Library
- pytorch
- License
- apache-2.0
- Created
- Sep 2, 2026
- Updated
- Sep 2, 2026