HoLo-6.5.1

HoLo 6.5.1 — byte-native multimodal research model (59M)

Proof-of-operation weights for HoLo 6.5.1: a tokenizer-free, byte-native, decoder-only prefix-LM (dim 512 / 8 layers / 8 heads, 59.4M params) running on the public non-learned 27-D HSL encoder (pip install hsl-embedding, MIT). Trained on a single RTX 4070. Not a benchmark-superiority claim — the release exists so every "it works" claim is reproducible.

• Live demo: https://holo-demo-p5txmh4dda-as.a.run.app
• Code: https://github.com/Woojiggun/holo-hsl
• Project card: https://huggingface.co/spaces/ggunio/holo-demo-space

Files

Grounding gap = extra bits/byte the model pays when its disk-retrieved facts are swapped for wrong ones (know_abl_bpb − know_bpb). It grew 0.001 → 1.835 across training: the model measurably READS its disk memory instead of memorizing (facts live in a disk store, patterns in weights).

Multimodal generation (mechanism, not quality): free-running 539-byte video windows ([256B frame | SEP | 256B mu-law audio | SEP | 24B caption | WSEP]) keep their structure markers unforced and place real English in the caption slot.

Usage

pip install "hsl-embedding>=0.5.0" torch
git clone https://github.com/Woojiggun/holo-hsl

from holo_generate import load, gen_text   # from the repo (Train/)
m, cfg = load("holo651_s3_multimodal_10k.pt", device="cuda")
out = gen_text(m, "The universe is ".encode(), n_new=120, temperature=0.7,
               origin_anchor=cfg["origin_anchor"])
print(out.decode("utf-8", "replace"))

Training data & why the license is NC

Because the S2/S3 stages include chat data that is partly model-derived / non-commercial, these weights are released for research use under CC-BY-NC-SA 4.0. Attribution: Korean Wikipedia (CC-BY-SA), Blender Foundation open movies (CC-BY).

Honest limitations

A newborn research model: text generation is rough, chat is shallow, generated frames/audio are toy quality (16×16 gray, 8 kHz mu-law). Free-running quality varies by checkpoint (see the repo notes on cosine-tail selection). No safety tuning of any kind — research use only.

Citation

Jinhyun Woo, HoLo: A Feasibility Study of Change-Rate-Based Multimodal Unification — DOI: 10.5281/zenodo.20581805