Qwen3.6-27B-GGUF

Qwen 3.6-27B — GGUF (iPad Pro / Mac edition)

GGUF quantizations of Qwen/Qwen3.6-27B, built and optimized for on-device inference on iPad Pro M-series and Apple Silicon Macs via llama.cpp or apps that wrap it (e.g. Haplo).

Built and quantized by jc-builds for the Haplo ecosystem. Original weights © Alibaba Cloud, redistributed under Apache 2.0 per the upstream license.

TL;DR

A 27B-parameter dense transformer from Alibaba's Qwen team, released in 2026. Top-tier on Arena open leaderboards in the dense ≤30B class — beats most older 70B models on reasoning, math, and code. Hybrid thinking / non-thinking modes (toggle via enable_thinking). 256K context. Apache 2.0 (no custom Qwen license restrictions).

Not for iPhone — too big. At 16 GB on disk and ~16 GB RAM at runtime, this targets iPad Pro (M-series, 16GB+ RAM) and Apple Silicon Mac.

Available quantizations

Q5_K_M was skipped — at this size Q4 is the practical phone/tablet pick and Q8 is the practical Mac pick. The middle quant adds storage cost without meaningful capability headroom on either device class.

Pick Q4_K_M for iPad. Pick Q8_0 only on Mac with plenty of RAM if you want near-FP16 reasoning quality.

⚠️ Quality note: This Q4_K_M was built via Q8_0 → Q4_K_M re-quantization (with --allow-requantize) instead of the standard FP16 → Q4_K_M path, because the FP16 intermediate (~54 GB) didn't fit on the build machine's disk. Expect a small (1-3% measurable) quality regression vs the canonical FP16-routed Q4_K_M. If you need maximum quality and have 110+ GB free, regenerate from FP16 yourself using the recipe below.

Performance on Apple Silicon

Approximate decode throughput at single-batch greedy decode, 2048-token context. Measured with llama-cli.

This model does NOT fit on iPhone. Even iPhone 16 Pro (8 GB RAM) cannot run a 16 GB model.

How to use

1. Haplo (iPad Pro / Mac)

The model appears in Haplo's model browser on Kuzco-1.1.0+ builds when running on a 16GB+ device. The download URL for Q4_K_M is:

https://huggingface.co/jc-builds/Qwen3.6-27B-GGUF/resolve/main/Qwen3.6-27B-Q4_K_M.gguf

2. llama.cpp (CLI)

Requires llama.cpp at commit ≥ f9cd456 (May 7, 2026) — needs the Qwen3_5ForConditionalGeneration / qwen35 arch register.

huggingface-cli download jc-builds/Qwen3.6-27B-GGUF Qwen3.6-27B-Q4_K_M.gguf --local-dir .

./llama-cli \
  -m Qwen3.6-27B-Q4_K_M.gguf \
  -p "Prove that the square root of 2 is irrational." \
  -n 1024 \
  --temp 0.6 \
  --top-p 0.95

For thinking-mode reasoning, prefix the prompt with /think (or set enable_thinking=True in your chat-template renderer). For fast chat, use /no_think.

3. Ollama

cat <<'EOF' > Modelfile
FROM ./Qwen3.6-27B-Q4_K_M.gguf
PARAMETER temperature 0.6
PARAMETER top_p 0.95
EOF
ollama create qwen3.6-27b -f Modelfile
ollama run qwen3.6-27b

Sampling defaults

For reasoning / math / code: temperature=0.6, top_p=0.95 (Qwen team's recommendation). For deterministic / structured: temperature=0.0, top_p=1.0.

The GGUF stores the upstream defaults — most clients (llama.cpp, Haplo, Ollama) will use them automatically.

Chat template

Qwen 3.6 uses standard ChatML, identical to Qwen 3.0–3.5:

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
{assistant}<|im_end|>

For thinking mode, the upstream template expects an opening <think>\n after the assistant turn. For non-thinking mode, a pre-closed <think>\n\n</think>\n\n block. Kuzco's .qwen3 formatter handles both via the enableThinking flag.

Quantization recipe

Built with llama.cpp at commit f9cd456 (May 7, 2026).

1. Downloaded Qwen/Qwen3.6-27B safetensors checkpoint (~55.6 GB BF16, 15 shards) via huggingface-cli.
2. Converted directly to Q8_0 via convert_hf_to_gguf.py --outtype q8_0 (skipping the FP16 intermediate to fit in available disk).
3. Re-quantized Q8_0 → Q4_K_M via llama-quantize --allow-requantize:

   llama-quantize --allow-requantize Qwen3.6-27B-Q8_0.gguf Qwen3.6-27B-Q4_K_M.gguf Q4_K_M
   

The canonical recipe is FP16 → Q4_K_M. We used the requantize path because FP16 didn't fit on disk. If you have 110+ GB free, regenerate from FP16 for slightly better quality.

Original model card

For full architecture, training, and benchmark details, see the upstream model card at Qwen/Qwen3.6-27B.

License

Apache 2.0, inherited from the original model. Commercial use, modification, and redistribution are permitted with attribution. See LICENSE for full terms.

Qwen 3.6 by Alibaba Cloud. Licensed under Apache 2.0.

Acknowledgements

• The Alibaba Cloud Qwen team for the Qwen 3.6 weights and an Apache 2.0 release.
• The llama.cpp team for the Qwen 3.5/3.6 architecture register that landed in the May 2026 cycle.
• The Haplo ecosystem this drop is built for.