Bonsai-27B-Brew-GGUF

☕ Bonsai-27B — Brew GGUF

Three standard K-quants of prism-ml/Bonsai-27B, self-quantized from their F16 reference on an NVIDIA GB10, for stock llama.cpp, Ollama, LM Studio, and any other standard GGUF runtime.

Bonsai-27B is a 27B-parameter dense model built on Qwen/Qwen3.6-27B (Alibaba Cloud, Apache 2.0). Prism ML's own GGUF repo focuses on extreme low-bit formats (1-bit, ternary) that need a custom llama.cpp fork with fused kernels. These three files don't — they're plain Q4_K_M / Q6_K / Q8_0, and load on any mainline llama.cpp build that already supports the qwen35 architecture.

The menu

Benchmarked with llama-bench -ngl 999 -p 512 -n 128 on a single NVIDIA GB10 (Blackwell, 121GB unified memory), full GPU offload, llama.cpp mainline build b10349.

Which one to pull

• Speed matters most, or you're VRAM-constrained: Espresso (Q4_K_M). ~53% faster generation than Cold Brew for 42% less disk, with no quality loss we could find on straightforward prompts.
• Default recommendation: Pour-Over (Q6_K). In our own spot-check (reasoning + a non-trivial coding prompt), it was the only one of the three that finished the harder prompt cleanly on the first try.
• Cold Brew (Q8_0) is the closest to full precision, but in our limited testing it didn't show a clear quality edge over Pour-Over — mostly you're paying in speed, not buying in accuracy.

Take that with real caution: it's a two-prompt spot-check, not a benchmark suite. Bonsai's thinking traces run long regardless of quant — a 700-token budget wasn't enough for any of the three to finish either test prompt; results above are all from a 2,600-token budget. Full writeup with charts: Bonsai Brew Ladder.

Usage

llama-cli -m Bonsai-27B-Q6_K.gguf \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  -cnv -st -n 2600

Bonsai is a reasoning model — every answer opens with a <think> block. Give it a generous -n; on non-trivial prompts it can use more of the budget on thinking than you'd expect, regardless of which quant you pick.

Verifying downloads

sha256sum -c SHA256SUMS --ignore-missing

Provenance & license

Quantized from prism-ml/Bonsai-27B-gguf (Bonsai-27B-F16.gguf) using llama-quantize from mainline llama.cpp — no re-training, no modification to the weights beyond standard quantization. Built from Bonsai-27B, Copyright 2026 Prism ML, Inc., itself built from Qwen3.6-27B, Copyright 2026 Alibaba Cloud. All Apache 2.0. Created using Bonsai by Prism ML.