KAT-Coder-V2.5-Dev-IQ2_M-GGUF

KAT-Coder-V2.5-Dev — IQ2_M GGUF

A single imatrix-guided IQ2_M quantization of Kwaipilot/KAT-Coder-V2.5-Dev, built to put a 34.7B mixture-of-experts coder inside roughly 11 GB.

What is and isn't verified

Verified: the file loads under llama.cpp and generates — prompt processing and token generation both exercised, exit code 0.

Not measured: there is no KL-divergence or perplexity table for this quant. IQ2_M is an aggressive tier and quality loss at ~2.7 bits per weight is real; this repo does not quantify it. If you want a quant release with a measured fidelity table, see Qwen3.6-27B-GGUF, where every tier is compared against the BF16 reference.

Treat this as "a working IQ2_M with its imatrix", not as a validated one.

Run it

llama-cli -m KAT-Coder-V2.5-Dev-IQ2_M.gguf -c 32768 -no-cnv -p "write a binary search in Rust"

-no-cnv matters: recent llama-cli builds otherwise drop into conversation mode and wait on stdin.

Note on metadata

The GGUF carries general.name = "Src", an artifact of the directory it was converted from. It is cosmetic and does not affect inference.

Credit

Model and weights © Kwaipilot — KAT-Coder-V2.5-Dev, Apache-2.0. This repo adds only the quantization. Weights otherwise unmodified.