KAT-Coder-V2.5-Dev-6bit-XL-mlx

leonsarmiento/KAT-Coder-V2.5-Dev-6bit-XL-mlx

KAT-Coder-V2.5-Dev by Kwaipilot — a 35B-A3B sparse MoE agentic coding model achieving 69.40 SWE-bench Verified, quantized for Apple Silicon using the BaseQuant_XL 6/8-bit recipe.

KAT-Coder-V2.5-Dev is a post-trained MoE built on Qwen3.6-35B-A3B via two-stage SFT (127K examples) + RL (10 epochs) with token-level consistency (TITO), truncated importance sampling (TIS), and hierarchical reward shaping from harness execution feedback. It ships text-only weights (no vision tower) despite the Qwen3_5MoeForConditionalGeneration architecture — the MLX conversion uses mlx_vlm with strict=False to skip the 333 missing vision_tower.* parameters.

About XL Quantization

BaseQuant_XL is a fully data-agnostic, static quantization. No calibration dataset, no sensitivity analysis, no importance matrix. Precision is allocated purely by architectural role — routing-critical layers get higher precision, bulk expert parameters get lower precision. The result is a transparent, faithful capture of the source model.

Data-dependent calibration quantizations (iMatrix, AWQ, GPTQ, oQ, oQ4e, etc.) use a calibration set to guide bit allocation. This can produce a skewed representation of the model: domains well-represented in the calibration data (English, popular topics, public or leaked benchmarks) are preserved better, while underrepresented domains (non-English languages, niche use cases, your own data) are preserved worse. XL avoids this trade-off entirely — it generalizes honestly because it is never fit to any particular data distribution.

Quickstart

pip install -U mlx-lm

python -m mlx_lm.generate --model leonsarmiento/KAT-Coder-V2.5-Dev-6bit-XL-mlx --max-tokens 256 --temperature 1.0 --top-p 0.95 --prompt "Implement an LRU cache in Python with O(1) get/put."

Works with LM Studio. Thinking mode is on by default (emits <think>...</think>). Pass enable_thinking=False for direct answers.

Quantization Strategy

BaseQuant_XL recipe — precision is allocated by layer importance, not applied uniformly:

• Bits per weight: ~6.9 · Total size: ~28 GB (6 shards) · Group size: 64

Recommended Inference Parameters

Thinking Mode (default)

Instruct Mode (non-thinking)

preserve_thinking is supported — set chat_template_kwargs: {"preserve_thinking": true} to retain thinking traces from historical messages (beneficial for agentic scenarios).

Chat Template

The chat_template.jinja and tokenizer_config.json were updated on 2026-07-30 to match the latest chat template published by Kwaipilot in Kwaipilot/KAT-Coder-V2.5-Dev. This update enforces that system messages must appear at the beginning of the conversation — mid-conversation system messages now raise an exception instead of being silently rendered.

Model Overview

MLX Benchmarks (n=50, instruct mode)

KAT-Coder leads Qwen3.6 on 4 of 5 benchmarks — the RL training for agentic coding shows in HUMANEVAL (+14 points). MMLU_PRO trails Gemma-4 significantly (64% vs 82%); KAT-Coder is specialized for SWE, not broad knowledge. Coding is strong but Gemma-4 remains overall champion.

MATHQA Thinking Efficiency (n=10, seed=42, 8192-token budget)

3-stage methodology: initial run → answer extraction (regex cascade) → re-test failures. Verified accuracy separates sampling/extraction variance from genuine capability gaps.

KAT-Coder is the reasoning efficiency champion — 9× less reasoning per correct answer than Gemma-4, and 20× less than Salience. Its SWE-focused RL training produces extremely terse, decisive reasoning chains. The one genuine failure (Q636) was an algebra error, not a reasoning deficit. KAT doesn't overthink — it computes, verifies, and answers.

Preserve Thinking

KAT-Coder is a Qwen3.6 model and supports preserve_thinking for agentic multi-turn workflows. Set chat_template_kwargs: {"preserve_thinking": true} to retain thinking traces from historical messages — essential for long-horizon SWE tasks where the model needs to recall its own reasoning from prior turns.

Verification (number-guessing game, 4 turns): With preserve_thinking enabled, KAT stays perfectly consistent — picking a secret number in turn 1 and giving correct higher/lower hints in all subsequent turns (1.4–1.6s each). The model reads its own prior <think> blocks rather than re-deriving the answer. Without preserve_thinking, the model re-picks a new secret number each turn and reconstructs consistency from visible conversation history — which works for simple games but risks drift in complex agentic tasks.

Source Model Benchmarks

From Kwaipilot's evaluation (vLLM/SGLang, BF16):

Paper: arXiv:2607.05471.

Capabilities

• Agentic coding — multi-step tool use, repo-scale edits, long-horizon SWE tasks
• SWE-bench SOTA — 69.40 Verified at this scale class
• Deep reasoning — structured <think>...</think> chains with one-token switch to disable
• Tool calling — RL-trained for reliable, non-degenerate tool-use patterns

Flight Simulator — Coded by This Model

A real-world coding benchmark: each model was prompted to write a complete flight simulator from scratch. The resulting code was rendered and recorded.

KAT-Coder made by Kwaipilot (KwaiKAT Team). MLX quantization by leonsarmiento.