ternary-bonsai-2-27b-coreai

Ternary Bonsai 2 27B — Core AI

A Core AI export of Bonsai 2 27B, PrismML's ternary build of Qwen3.8-27B. The bundle runs on an Apple silicon Mac through the native bonsai-swift host with no Python in the generation loop.

The repository contains:

• a portable Core AI .aimodel with 64-token and 16-token prefill functions plus single-token decode;
• an h16s ahead-of-time compile for the M4 Pro;
• the Qwen tokenizer and chat template; and
• bundle metadata and SHA256SUMS.

Requirements

• Apple silicon Mac running macOS 27
• Xcode 27 toolchain and Swift 6.4 or newer
• approximately 14 GB of disk space
• bonsai-swift

The published measurements used an M4 Pro, macOS 27.0 build 26A428, Xcode 27.0 build 27A5237l, and Apple Swift 6.4. The included h16s asset is specific to the M4 Pro. Other Apple silicon architectures require an AOT compile produced from the portable .aimodel.

Download and run

hf download rahulrachuri/ternary-bonsai-2-27b-coreai \
  --local-dir bonsai2_27b_decode_pq2_0_pf64

git clone https://github.com/RahulRachuri/bonsai-swift.git
cd bonsai-swift
swift build -c release

export BONSAI_BUNDLE="$PWD/../bonsai2_27b_decode_pq2_0_pf64"
.build/release/bonsai-swift chat --prompt "The capital of France is" --new 64

The host does not download model artifacts implicitly. Applications must provide the bundle path through --bundle or BONSAI_BUNDLE.

Bundle contract

main      input_ids [1,1]   position_ids [1,?] -> logits [1,1,V]
prefill   input_ids [1,64]  position_ids [1,?] -> logits [1,64,V]
prefill16 input_ids [1,16]  position_ids [1,?] -> logits [1,16,V]
states    keyCache, valueCache, convState, recState

The exported context limit is 4,096 tokens. The host allocates a KV capacity between the traced minimum of 2,048 and that limit. Generation is greedy.

Validation

The bundle was teacher-forced against greedy reference decodes from PrismML's MLX runtime. The host uses the same 64-token, 16-token, and single-token planner for validation and generation.

The three position mismatches occur inside the chat template's system prompt where the MLX reference has fp16-scale top-two margins of 0.002 to 0.024. Graph-provided and host-computed argmax agreed on all 134 walked fixture steps.

A separate ten-prompt battery covered 4,405 positions across prose, code, numbers, newlines, CJK, mixed scripts, emoji, and repeated-token stress cases. Chunked prefill and an S=1 walk agreed at 4,347 positions and produced 80/80 identical continuation tokens. Each case ran in a fresh process; resident memory remained near 7.5 GB without additional swap growth.

Performance

M4 Pro, release build, Python-free generation loop:

Provenance

• Language weights: prism-ml/Ternary-Bonsai-2-27B-gguf, revision 6ed5e12bf84b7a63069882c91dd9e9218647d17b, file Ternary-Bonsai-2-27B-PQ2_0.gguf
• Tokenizer and chat-template lineage: Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
• Compression: ternary group-128 PQ2_0 with signed blockwise Walsh-Hadamard transforms
• Export tools: coreai-core 1.0.0b2
• M4 Pro AOT compiler: coreai-build-3600.82.1
• Conversion source commit: c9e9e6b

The conversion and validation source is maintained in the community Core AI model zoo. The portable graph was exported on 2026-09-18. The full file manifest is recorded in SHA256SUMS.

Limitations

• The language export is text-only; PrismML's optional vision projection is not included.
• The exported context limit is 4,096 tokens.
• Generation in bonsai-swift is greedy.
• The included AOT asset targets the M4 Pro. The portable source graph is included for producing architecture-matched AOT assets.

License

Apache License 2.0. The converted artifact inherits the Apache-2.0 terms of the PrismML and Qwen sources. See NOTICE for attribution. Created using Bonsai by Prism ML.