Qwen3.8-27B-Ternary-Bonsai-2-DFlash2-MLX

Qwen3.8-27B-Ternary-Bonsai-2-DFlash2-MLX

Prism ML's ternary Ternary-Bonsai-2-27B build of Qwen/Qwen3.8-27B, repacked for chad, a Claude-Code-style local coding agent for Apple Silicon, with its speculative decoder bundled in. This is chad's default model. Created using Bonsai by Prism ML.

One download gets both halves: the ternary target model and the block drafter it decodes with, already quantized. Nothing is built on first run.

What the weights are

Every projection of Qwen3.8-27B (a dense qwen3_5 hybrid: 64 layers, 48 GatedDeltaNet + 16 full attention) is stored in a Hadamard-rotated basis: multiplied by a fixed sign vector and put through a blockwise Walsh-Hadamard transform offline, then quantized to 2-bit affine group-128 whose three levels reproduce the ternary set {−s, 0, +s}. The rotation costs no extra bits and no extra weight traffic, but the matching transform must be applied to the activations at runtime and inverted after the embedding lookup. An ordinary MLX affine loader finds tensors of exactly the right shapes, skips both transforms, and produces plausible garbage without raising. chad routes the pack on its model_type (prism_hadamard_qwen35) to a loader that does the rotation; mlx-lm alone will not run it correctly.

• Footprint: 7.15 GB of weights + a 1.1 GB bundled drafter; 262k native context.
• RAM: built for 24 GB Apple Silicon. chad sizes its context window from the live Metal budget: on a 24 GB M4 Pro the compaction threshold lands around 150k tokens, against ~30k on the 3-bit quant this replaces.
• Speed (M4 Pro, greedy, 512-token prompt, 128-token decode): 21 tok/s serial, 64 tok/s with the drafter (94% of drafted tokens accepted); prefill ~99 tok/s.
• Quality: teacher-forced NLL on code is unchanged against the pack's own fp32 forward (1.502 vs 1.500); chad's private core and languages eval tiers pass 13/13 and 17/17, the same as the 3-bit quant.

What differs from the Prism pack

• Text only. The pack's FP16 vision tower (0.92 GB) is dropped; config.json says so (components.vision: false). The language-model tensors, sign vectors and hadamard.json are byte-identical to the source revision recorded in the safetensors header (source_revision).
• Tokenizer and chat template from the base model. The pack's tokenizer.json has the same vocabulary and added tokens but different merges and pre-tokenizer, which trips transformers' regex warning on every load and diverges on combining marks. This repo ships the base tokenizer, and the chat template with reasoning_effort defaulting to medium (the pack's says xhigh; upstream Qwen's says xhigh). Pass reasoning_effort explicitly to override it.
• config.json carries the base checkpoint's transformers_version. Without it, transformers' Mistral-regex heuristic treats any checkpoint as a legacy Mistral tokenizer and warns on every load that tokenization is wrong; the key is what the base Qwen3.8-27B config carries, and the tokenizer files here are that checkpoint's.
• No bundled runtime. The pack ships Python it asks callers to sys.path.insert; chad reimplements the forward and does not import code out of a model download.
• dflash/: the DFlash2 block drafter, identical to the one bundled with nathansutton/Qwen3.8-27B-UD-Q3_K_XL-DFlash2-MLX.

The bundled DFlash2 drafter (dflash/)

A 1.9B model that reads the target's residual stream at five tapped layers and proposes a whole block of tokens in one forward, quantized to 4-bit group-64 in MLX format. A drafter reads the residual stream, which the weight quantization only perturbs, so the drafter trained against the bf16 base transfers to the ternary target unchanged. chad verifies the block in one batched target forward and accepts by exact rejection sampling, so greedy output is token-identical to unspeculated decoding (to kernel rounding) and sampled output keeps the model's true distribution at any temperature.

It is deliberately in a subdirectory: mlx-lm globs model*.safetensors at the repo root, so dflash/model.safetensors is invisible to the base loader.

The drafter weights are converted from the DFlash2 release for this model (z-lab/Qwen3.8-27B-DFlash2); the method is Chen et al., DFlash: Block Diffusion for Flash Speculative Decoding (arXiv:2602.06036). Only the quantization and the MLX layout are ours; please honor the original release's license and citation terms.

Use it

uvx chad-code            # this is the default model; first run downloads it (~8 GB)

License and attribution

Apache-2.0. The ternary weights are copyright Prism ML, Inc., built from Qwen3.8-27B (copyright Alibaba Cloud, Apache-2.0); LICENSE and NOTICE.txt are the pack's own. Created using Bonsai by Prism ML.