Ternary-Bonsai-2-27B-DFlash2-ft5-mlx-4bit

Ternary-Bonsai-2-27B-DFlash2-ft5-mlx-4bit

The prequantized form of Ternary-Bonsai-2-27B-DFlash2-ft5, a DFlash 2 drafter fine-tuned against PrismML's Ternary-Bonsai-2-27B. Same weights, not a new training run. The bf16 drafter is quantized to 4 bits every time a runtime loads it. This repository holds the result of doing that once, so the download is 2.83 times smaller. Every measurement on the bf16 card applies unchanged: the bits that reach the GPU are the same.

Load it

Both loader checks ran on this upload's config.json and model.safetensors; their records are in the bonsai2-drafter repository's evidence/prequantized-4bit/second-loaders/.

Plain mlx_lm.load(path) is not expected to work, since the config's model_type is the drafter architecture, and the bf16 file behaves the same way.

Validation

• Tensor equality. The saved artifact, loaded through the patch, equals the bf16 drafter quantized at load time: every parameter name, shape, dtype and value, the module classes, the quantization parameters and the DFlash 2 configuration. That holds in-process and in a fresh process that cannot read the source directory (271 checks). The exporter itself refuses to publish an artifact that does not reload identically. That check is the repository's bench/tests/test_dflash_prequantized.py --source … --artifact …; its log is not in the evidence bundle.
• Served equivalence. On the frozen prompt suites, the bf16 and 4-bit packagings produced the same served loop: the same tokens, finish reasons, token counts, rounds and round lengths (contract artifact-equivalence/v1). HTTP decode ran ABBA at a throughput ratio of 1.0001, with no newly failing quality fixture and 56/56 tool calls in both arms. Those runs used the same model.safetensors bytes under a different metadata layout. This release changes only config.json, and the tensor file is byte-identical. The records are in the repository's evidence/prequantized-4bit/.
• One Apple M4 Pro, 48 GB. No speed claim is made for this packaging beyond "the same as bf16".

Limitations

Those of the bf16 drafter apply, including the disclosed non-termination of one quality prompt at an 8-bit KV cache, which is a property of the cache setting and not of the drafter. There is no 4-bit convention for DFlash 2 drafters in common use. This one follows mlx-lm's, keeps the codebooks in bf16, and says so in its config.

License

Apache-2.0, with the bf16 release's LICENSE and NOTICE, which carry Prism ML's notice and its requested attribution: "Created using Bonsai by Prism ML." This is an independent release, not an official Prism ML, Qwen, Inco AI or z-lab one.