Ternary-Bonsai-27B-mlx-lossless-1.75bpw

Ternary Bonsai 27B — MLX, lossless, 1.75 bits/weight

The smallest exact representation of Ternary Bonsai 27B on Apple Silicon: the same ternary weights as the official MLX 2-bit distribution, stored at 1.75 bits/weight instead of 2.25 — exactly 7/9 the size — and executed directly from packed storage by custom Metal kernels. No weight is ever dequantised into a model-sized buffer. Under greedy decoding it emits token-for-token identical output to the official checkpoint while holding 22.19% less live memory (5.89 GB against 7.57 GB during generation).

This is a memory-density result, not an accuracy or speed win. The official 2-bit checkpoint already stores exact ternary weights (its affine parameters satisfy bias == -scale with codes in {0, 1, 2}), so there was no accuracy gap to close — both checkpoints hold the same weights, verified elementwise. Token generation runs at 0.89–0.96× the official checkpoint's rate and prompt processing at 0.64–0.87×, measured on an M4 Max; if you want maximum speed and have the memory, use the official 2-bit repo. If you want the smallest exact Bonsai, this is it.

Run it

Requires an Apple Silicon Mac with 16 GB+ unified memory (the weights are 5.5 GB on disk and generation holds ~5.9 GB live) and Python with mlx-lm >= 0.31.3:

pip install mlx-lm
mlx_lm.generate --model inductiveML/Ternary-Bonsai-27B-mlx-lossless-1.75bpw \
  --prompt "Explain what a ternary weight is." --max-tokens 256

The first run downloads the checkpoint (~5.5 GB) into the Hugging Face cache; later runs load from cache. From Python:

from mlx_lm import load, generate

model, tokenizer = load("inductiveML/Ternary-Bonsai-27B-mlx-lossless-1.75bpw")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain what a ternary weight is."}],
    add_generation_prompt=True,
    tokenize=False,
)
text = generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=True)

mlx-lm newer than 0.31.3: upstream has merged a trust_remote_code gate around checkpoints that carry their own architecture code, as this one does. On releases that include it, add --trust-remote-code to the CLI (or trust_remote_code=True to load(...)). On 0.31.3, the current release, no flag is needed.

The numbers

Measured on a MacBook Pro M4 Max (40 GPU cores, 48 GB), macOS 26.5.2, MLX 0.32.0, mlx-lm 0.31.3, against prism-ml/Ternary-Bonsai-27B-mlx-2bit running through stock mlx-lm. Full methodology, paired bootstrap confidence intervals, and every gate's machine-readable result are in the Ternel repository.

Intelligence density

The Bonsai model card measures capability per deployed byte as intelligence density — D = -log2(1 - score/100) / size_GB — and quotes 0.400 /GB for Ternary Bonsai 27B from a 5.9 GB size. That 5.9 GB is the format's ideal size, which the same card notes no shipped build reaches — "the deployed footprint sits above the representation's information-theoretic minimum until native ternary kernels close the gap": 7.17 GB deployed in llama.cpp (2.125 bits/weight slots), 8.49 GB in the official MLX distribution (2.25 bits/weight).

This repo is that gap closed, on Apple Silicon. The packed artifact deploys at 5.89 GB — the ideal size, as a runnable checkpoint rather than a limit:

* Scores are the official Bonsai card's 15-benchmark thinking-mode averages, not re-run here. For this repo that inheritance is exact rather than approximate: the language-model weights are bit-identical to the official checkpoint (verified over all 26,893,352,960 — see Fidelity) and greedy output is token-identical, so the official score is this artifact's score; the density gain is entirely the denominator. Two of the 15 benchmarks exercise the vision tower, which no Bonsai language-model artifact ships (theirs load a separate mmproj pack; this repo is text-only).

Fidelity

TQ1_G128 stores each group of 128 ternary weights in 28 bytes: two raw FP16 scale bytes, 25 base-3 bytes of five trits each, and a tail byte of three. The conversion is a re-encoding, not a re-quantisation:

• All 26,893,352,960 ternary weights across 498 quantised tensors verify bit-exactly against the pinned source GGUF — zero mismatches, and zero raw scale-bit mismatches across all 210,104,320 groups.
• Teacher-forced logits agree with the official checkpoint at every one of 343 positions tested (argmax-identical; max KL 4.8×10⁻⁵ nats, residual differences are kernel accumulation order, not representation).
• Greedy generation is token-identical across every test prompt, 576/576 tokens.

This checkpoint contains code

config.json names a model_file, so mlx-lm builds the model from ternel_packed_model.py shipped in this repo — that is how the packed weights run without installing anything beyond mlx-lm. You are trusting that file, so it is built to be audited:

• It is a single self-contained Python file that imports only math, mlx, and mlx_lm, generated mechanically from the sources at inductiveML/ternel — never hand-edited, and a test in that repo fails if the shipped file drifts from what the sources emit.
• ternel_manifest.json records its SHA-256 and the SHA-256 of each source module it was generated from, alongside per-tensor content hashes for every weight in the checkpoint.
• The model class subclasses mlx_lm.models.qwen3_5.Model; the file adds the packed storage layout, three Metal kernels (single-vector, batched, and embedding lookup), and the module replacement — nothing else.

Compatibility

• Supported: mlx-lm ≥ 0.31.3 on Apple Silicon (CLI, Python API, and anything that calls mlx_lm.load with checkpoint code enabled). Verified against mlx 0.32.0 / mlx-lm 0.31.3.
• Not supported: LM Studio. Its MLX engine deliberately does not execute code shipped in checkpoints, so it refuses any repo that declares a model_file. Native support would require the packed format landing in mlx-lm itself.
• Text-only. The source GGUF carries no vision tower, and config.json sets language_model_only: true. Bonsai's vision path is not in this checkpoint.

Sources, pinned

Every weight traces to a pinned upstream artifact; both hashes were verified after download, not trusted:

License and attribution

Apache 2.0, inherited from the sources; LICENSE.txt and NOTICE.txt ship in this repo. Created using Bonsai by Prism ML. Bonsai is built from Qwen3.6-27B, Copyright 2026 Alibaba Cloud, also Apache 2.0. This repository is an independent re-packing and is not affiliated with Prism ML or Alibaba Cloud.