Downloads · 30 days
570
33% of all-time downloads
Schiltmans/Ternary-Bonsai-2-27B-DFlash2-ft5-GGUF
Ternary-Bonsai-2-27B-DFlash2-ft5-GGUF is a machine learning model from Schiltmans. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
GGUF files of Ternary-Bonsai-2-27B-DFlash2-ft5, a DFlash 2 drafter fine-tuned against PrismML's Ternary-Bonsai-2-27B, for llama.cpp's DFlash 2 speculative decoding (--spec-type draft-dflash). Same weights, not a new t…
Downloads · 30 days
570
33% of all-time downloads
All-time downloads
1.7K
Public
Repo size
3.9 GB
Likes
4
Public
Click a slice to open those files.
.gguf3.9 GB · 100%
From the Hugging Face model README
GGUF files of Ternary-Bonsai-2-27B-DFlash2-ft5,
a DFlash 2 drafter fine-tuned against PrismML's Ternary-Bonsai-2-27B, for llama.cpp's DFlash 2 speculative decoding
(--spec-type draft-dflash). Same weights, not a new training run.
| File | Size | Draft acceptance | sha256 |
|---|---|---|---|
Ternary-Bonsai-2-27B-DFlash2-ft5-Q8_0.gguf | 2,056,415,104 bytes | 0.432 | 70a7c6be7b3ec7ae5427e51cb29e31663d2759a1f81df4694c098922ff9f99c9 |
Ternary-Bonsai-2-27B-DFlash2-ft5-Q4_K_M.gguf | 1,143,007,104 bytes | 0.427 | f63a94fedab264f65213496b140eb6f88e78046768effafc86ee3c3b7e9d013a |
Ternary-Bonsai-2-27B-DFlash2-ft5-Q2_K.gguf | 705,431,424 bytes | 0.398 | eaeff9506d616264a15ca16bed840807d99b56f0afb17227ef6f8890284e698e |
The stock drafter's Q8_0 (z-lab/Qwen3.8-27B-DFlash2-GGUF) accepts 0.395 under the same conditions (see Validation).
Q4_K_M keeps almost all of the Q8_0's acceptance at 56% of its size; Q2_K matches the stock Q8_0's acceptance at a third of the size.
The tensor layout matches z-lab/Qwen3.8-27B-DFlash2-GGUF: the same 81 tensor names and the same non-descriptive
metadata, so these files should load wherever the stock drafter's GGUF does.
All three were made with PrismML's fork at prism-b10754-2459f68. The Q8_0 comes straight from convert_hf_to_gguf.py --outtype q8_0 and is byte-identical to this repository's original Q8_0. Q4_K_M and Q2_K were converted to BF16 first,
then quantized with that build's llama-quantize, without an importance matrix: a DFlash drafter cannot run on its own
over text, so llama-imatrix has nothing to measure. The tokenizer is Qwen/Qwen3.8-27B's, whose tokenizer.json is
byte-identical to the Bonsai pack's. The source is the bf16 drafter at revision 2a2c2c1e, weights sha256
63399215…55b3.
It needs a llama.cpp build that has both DFlash 2 and PrismML's support for the target's Hadamard-rotated ternary pack.
PrismML's fork (PrismML-Eng/llama.cpp) has both since release
prism-b10743, which merged PrismML-Eng/llama.cpp#261. Upstream
llama.cpp has DFlash 2 but could not load the target when this card was first written (2026-09-24); that has not been
retested.
llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf -md Ternary-Bonsai-2-27B-DFlash2-ft5-Q4_K_M.gguf \
--spec-type draft-dflash --spec-draft-n-max 7 -ngld 99 -ngl 99 -fa on --jinja
Use it on CUDA, not on a Mac. On Metal the fork's DFlash 2 path is still slower than no drafter at all: 7.1-8.0 tok/s with any of these drafters against 20.7 tok/s without one (Validation). On a Mac, use the bf16 or 4-bit drafter with mlx-dspark, or MTPLX without a drafter, which the bf16 card shows is faster still. This author has not measured the GGUFs on CUDA.
One Apple M4 Pro, PrismML's fork at prism-b10754-2459f68 (Metal), the target Ternary-Bonsai-2-27B-PQ2_0.gguf, the
flags above with a 32,768-token window, and the repository's bench5.py (five prompts, three repetitions, greedy). Each
arm is a fresh server; draft acceptance is from the server log.
| Drafter | Draft acceptance | Decode, tok/s | Tokens generated |
|---|---|---|---|
| none | 20.70 | 5,706 | |
| ft5 Q8_0 | 0.432 | 7.92 | 5,706 |
| ft5 Q4_K_M | 0.427 | 7.88 | 5,706 |
| ft5 Q2_K | 0.398 | 7.36 | 5,706 |
| stock Q8_0 | 0.395 | 7.10 | 5,706 |
| ft5 Q8_0, repeated last | 0.432 | 7.96 | 5,706 |
Every arm generates the same 5,706 tokens: greedy speculative decoding returns the target's own output, so the drafter changes speed, not text. The rule fixed before the run was to offer a quantization only if its acceptance beat the stock drafter's Q8_0 on this build. Both did, Q2_K by a small margin (0.398 against 0.395). The first release's diagnostic (2026-09-24, a patched build of the fork) measured 0.405 against the stock drafter's 0.346; this build accepts more for both.
The records and per-tensor hashes of the first release are in the repository's
evidence/gguf-fork/.
Apache-2.0, with the bf16 release's LICENSE and NOTICE, which carry Prism ML's notice and its requested
attribution: "Created using Bonsai by Prism ML." This is an independent release, not an official Prism ML, Qwen, Inco
AI or z-lab one.