hlm5-1b-trunk

HLM5 — 1B frozen trunk for certified knowledge editing

HLM5 is a frozen decoder-only language model paired with a certified, auditable, exact-key edit layer. It is the trunk for "Certified Knowledge Editing: Closed-Form Reachability Guarantees for Additive Edits to a Frozen Language Model" (Marius Dima, Qriton). A slot memory attaches to this frozen trunk; a hidden-state query selects slots through a hard cosine gate and adds the value to the pre-head representation. Because the gate and read see only the unperturbed hidden state, the post-edit logits are affine in the edit strength, so single-token reachability is one-dimensional linear feasibility — solved in closed form as a certificate that accepts or refuses each edit before injection and doses the accepted ones at a certified strength.

• Code, scripts, and result artifacts: https://github.com/qriton/hlm5
• Paper: in the repository (paper/); arXiv link forthcoming.

Model description

• Architecture: decoder-only transformer, dim 1792, 24 layers, 14 heads, context length 1024.
• Vocabulary: 65,536-token byte-pair tokenizer (fineweb-65536-compat.json).
• Training data: FineWeb (~20B tokens).
• Validation PPL: 18.01 (baseline trunk).
• Parameters: ~1.045B.

This repository hosts hlm5_lm_baseline_fineweb_g3_final.pt (the frozen baseline trunk, step 150,999), hlm5_lm_hybrid_fineweb_g3_final.pt (the matched co-trained-memory arm, step 149,999), and the tokenizer.

Usage

Use the package loader from the GitHub repository. It loads checkpoints with torch.load(..., weights_only=True), freezes the trunk, and pairs the checkpoint with the bundled tokenizer:

from hlm5.io import load_trunk, load_tokenizer

model, ck = load_trunk("baseline")
tok = load_tokenizer()

Avoid older snippets that call torch.load(..., weights_only=False) directly.

Results

Evaluated as the deployed certificate-governed pipeline (whitened degree-5 gate, threshold 0.95) on a 17-fact set; every number traces to a JSON in the GitHub repository under results/.

• Certificate-dosed efficacy 0.765 (13/17), rising to 0.882 (15/17) with residual-synthesis rescue through the full memory path, at locality 1.000 (bit-identical logits on neutral prompts) and paraphrase transfer 0.020 (exact-key by design). Synthesis rescues two of four otherwise unreachable facts; the other two are refused rather than misclassified. The 0.529 figure some earlier material led with is the global-boost dosing ablation — a single strength that overshoots the certified interval — not the method.
• Operating envelope: 78.4% of 1,200 single-token targets reachable at a novel-entity key; 79.4% ± 2.2% across 60 keys.
• Gate: perfectly separable (exact-key 1.0, all paraphrase/typo/other-relation 0.0); realized own-slot weight 0.9993.
• No-tax: matched baseline-vs-hybrid validation-PPL delta 0.116% at 1B (0.018% at 136M).

Implementation note: the current certificate engine uses float64 arithmetic and exact slope signs (EPS = 0.0), with a regression test for near-zero negative slopes. The 2026-08-10 registered refresh replaced the older unbound artifacts; the promoted JSONs bind the immutable preflight, producer, model/data inputs, and comparison verdict. Run python scripts/verify_artifacts.py before publishing or redistributing them.

Intended use

Research on certified, auditable knowledge editing: reachability certificates, certificate-governed dosing, exact-key slot memories, and edit/forget/rollback receipts on a frozen trunk. Suitable for reproducing the paper and building on the certificate machinery.

Out of scope

This is a research base model, not an instruction-tuned or chat model. It is not aligned, not safety-tuned, and not intended for deployment as an assistant or for generating end-user-facing text. The edit layer is exact-key by design: it does not generalize edits to paraphrases.

Citation

@article{dima2026certified,
  title   = {Certified Knowledge Editing: Closed-Form Reachability Guarantees
             for Additive Edits to a Frozen Language Model},
  author  = {Dima, Marius},
  year    = {2026},
  doi     = {10.5281/zenodo.21258598},
  note    = {arXiv link forthcoming}
}

License

Business Source License 1.1 (BSL 1.1), © 2026 Qriton Technologies S.R.L. Academic research, education, personal, and non-commercial evaluation use is permitted; commercial use is not. Converts to Apache-2.0 on 2030-01-01. See the LICENSE file in the GitHub repository; commercial inquiries: [email protected].