Bonsai-2-27B-CRACK-PTQ1-MLX

Bonsai 2 27B CRACK, compact PTQ1_0 for MLX

This is a text-only MLX safetensors repack of the PTQ1_0 GGUF from dealignai/Bonsai-2-27B-1bit-CRACK-GGUF. The 402 ternary matrices retain their original 28-byte-per-128-weight PTQ1_0 blocks; they are not converted to MLX affine 2-bit weights. A separate mlx-kquant Metal operator multiplies those packed blocks during inference. The model.safetensors file is 5,994,496,877 bytes (5.99 decimal GB); it is 47,847,949 bytes larger than the source GGUF. The package also includes a 12 MB tokenizer and small runtime files.

This is a packaging and runtime integration of existing CRACK weights and existing packed Metal operators, not a new compression algorithm or model. Conversion source and verifier: Obscyra-app/bonsai-crack-ptq1-mlx. There is already a CRACK 1.75-bit JANG MLX release with a different packed codec and vMLX runtime. This package preserves the PTQ1_0 representation of the GGUF and exposes a small local text API.

Requirements

• Apple Silicon Mac, macOS 26.2 or newer, Python 3.10–3.14.
• mlx-kquant==0.4.13 (which pins mlx==0.32.1), mlx-lm==0.31.3, NumPy, tokenizers and Jinja2; install with pip install -r requirements.txt.
• Enough unified memory for the 6 GB weights, activations and KV cache. Tested on MacBook Air M3 with 16 GB unified memory.

Run

Download this complete repository with the official Hugging Face CLI, then run from the downloaded directory:

hf download Nurymanau/Bonsai-2-27B-CRACK-PTQ1-MLX --local-dir bonsai-crack-ptq1-mlx
cd bonsai-crack-ptq1-mlx
pip install -r requirements.txt
python3 quickstart.py 'Сколько будет 2+2? Ответь только числом.'
python3 server.py --host 127.0.0.1 --port 8091

The server accepts text-only, non-streaming, deterministic POST /v1/chat/completions requests and supports GET /v1/models. It defaults to loopback access. Example:

curl http://127.0.0.1:8091/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"bonsai-crack-ptq1-mlx","messages":[{"role":"user","content":"2+2?"}],"temperature":0,"max_tokens":16}'

This custom PTQ1_0/Hadamard schema is not loadable by stock mlx_lm.load(). The included quickstart.py and server.py are the supported entry points. Vision inputs, streaming, sampling, LoRA and long-context quality are not validated in this text package.

Reproduction and checks

Source GGUF revision: 9665897ee63152226ecd64a1036d14292397186f. Source file SHA256: dcca61238e280432c4ce2d4c6c1d61214cbcb8a5d93ed98dea20d8bcef385a42. This package's model.safetensors SHA256: e6815083fbd8d0945840f66ed0299be89dcaef4094b6fccc4c5cfdadb27dbb9b. An independent verifier checked all 402 packed matrices byte-for-byte against the source GGUF after the required model row reordering. See build-manifest.json and the project builder bonsai_crack_compact_build.py for the exact repacking path.

On the tested M3/16 GB Mac, fresh package load took 2.19 s; four short greedy text prompts produced the same visible answers as the source GGUF: 4, Paris, 6, and Привет. This small smoke check does not establish general capability or full numerical equivalence. The measured decode rate on the three-token Привет response was about 4.0 tokens/s, versus roughly 5.1–5.4 tokens/s from the PrismML GGUF Metal server on these short prompts.

Credits and license

Model: Prism ML Bonsai 2, derived from Qwen3.8-27B. CRACK weights: dealignai. Packed Metal operator: asher/mlx-kquant, MIT licensed, installed as a dependency. The Prism ML license and notice are included. Created using Bonsai by Prism ML. The weights are distributed under the source model's Apache 2.0 license. This repack does not claim ownership of the base model, CRACK modification, or mlx-kquant kernels.