Bonsai-27B-mentria

Bonsai-27B-mentria

prism-ml's Bonsai-27B — a natively 1-bit (quantization-aware-trained) 27B model — repacked into mentria-q1g128-v1 safetensors for mentria-engine, a custom WebGPU runtime that runs the model entirely in your browser. 27 billion parameters in 3.79 GB, with an optional vision tower (+741 MB) for image understanding — validated end-to-end in-engine (image → ViT → M-RoPE prefill → answer).

This is the top tier of the mentria model ladder: the Qwen3.5-VL small tiers (0.8B → 2B → 4B) cover phones and laptops; Bonsai-27B serves Apple-Silicon-class GPUs and ≥6 GB discrete GPUs. Same architecture family top to bottom (Bonsai-27B is a 1-bit compression of Qwen3.6-27B — hybrid Gated-DeltaNet + attention with the stock Qwen3-VL vision tower).

Try it live:

• mentria.ai/tools/ai-chat — text + vision chat; the 27B tier is offered on Apple-silicon Macs and computers with a dedicated graphics card
• Run an LLM in your browser — live demo, download sizes, measured speeds and browser support

Files

Format: mentria-q1g128-v1

Every weight is one bit (0 → −scale, 1 → +scale) with one FP16 scale shared per 128-weight group — ≈1.13 bits/parameter effective (1.125 in the quantized tensors, 1.128 across the whole file). The code words are byte-identical to the upstream GGUF Q1_0 (g128) tensors; only the container changes (safetensors with a scales header, split for the browser's per-buffer limits). Bonsai is QAT — born 1-bit, not post-hoc quantized — so there is no precision loss relative to the published model.

Correctness

Every kernel on every shipping route is teacher-forced-golden-certified against the PrismML llama.cpp reference: 192/192 forced positions, zero decisive disagreements, across 4 engine configurations (including the no-subgroups fallback used by Safari/iOS). Greedy engine output is token-identical to the reference runtime.

Quality (measured s1836 via the PrismML llama.cpp reference runtime — the engine is token-identical to it at greedy)

Likelihood-based scores under-report instruction-tuned models relative to generative protocols; use the generative row for cross-model comparisons.

Independent verification of the upstream "≈90% of FP16" claim (2026-08)

We reproduced prism-ml's published 1-bit thinking-mode numbers on this exact repack (their protocol: thinking on, temperature 0.7, top-p 0.95, top-k 20, EvalScope prompt formats; seed-42 subsets; scoring by rule-based answer equivalence + hand review):

A 20-question greedy spot-check through the WebGPU engine matched the reference runtime on every chain that fit the harness context — zero completed-chain divergences. Scope note: the upstream "90%" is an average over 15 benchmarks; retention on agentic and strict instruction-following categories (τ²-Bench, IFBench) is materially lower than on the math/knowledge benchmarks shown here — see the upstream whitepaper before relying on those behaviors.

Where the "≈90%" holds, and where it does not (2026-09-25)

The upstream figure is an average over standard benchmarks. To show what it means in practice, we compared the same parent model at three precisions on 149 harder questions (niche lists, long-tail facts, multi-step reasoning, professional topics, long-form writing, strict formats, everyday chat and code), asked in English (82) and in Japanese (67). All three used greedy decoding, thinking off, the same system prompt and a 1,500-token cap. One reviewer per language graded the three answers to each question blind (answer order shuffled per question): 0 = wrong or unusable, 1 = partly right, 2 = right and complete.

What this means in practice:

• English everyday use holds up well. On everyday chat, long-form writing and format-following questions, this model reaches about 75% of the full-precision model's grade.
• Other languages are hit harder. In Japanese the same categories reach about 26%: expect repetition loops, invented technical terms and unreliable formal register (keigo). In Spanish, Portuguese and French, the model's answers were worse than its own English answers to the same question on roughly half of these questions.
• Rare knowledge suffers most, in every language. Exhaustive lists and long-tail facts are often invented, sometimes followed by a repetition loop. For such questions, verify against a primary source.
• Scope. These are small, deliberately hard samples with one reviewer per language; they complement the standard benchmarks above rather than replace them. The full-precision answers came from the unquantized Qwen3.6-27B served on 2× H100; the 1-bit and ternary answers came from local runs.

Performance

Model details

About mentria.ai

Mentria is AI that runs in your browser, with built-in tools, an extensions library and card decks for learning. Add your own extensions and decks. Your data stays private.