KAT-Coder-V2.5-Dev-VL-oQ8e-mtp

KAT-Coder-V2.5-Dev-VL-oQ8e-mtp

Kwaipilot/KAT-Coder-V2.5-Dev with its vision tower and MTP head restored from the base model it was fine-tuned from, quantized to 8-bit for Apple Silicon.

Kwaipilot's open-weight release ships language-model weights only — the vision components and the multi-token-prediction head were stripped. Both are recoverable, because KAT's config.json is byte-identical to Qwen/Qwen3.6-35B-A3B's except for one field (mtp_num_hidden_layers: 1 → 0), and the text architecture is unchanged. This build puts them back.

Read this first: what you need to load it.

The oQ8e quantization is not a proprietary format. oQ writes standard mlx-lm safetensors with a standard per-module quantization config, and portable oQ models load in stock MLX apps — confirmed here by loading an oQ4e checkpoint under stock mlx-vlm with omlx never imported.

What makes this repo oMLX-only is the restored MTP head. Stock mlx-vlm 0.6.3 does a strict load and rejects the 42 extra language_model.mtp.* tensors: ValueError: Received 42 parameters not in model. So as published this needs oMLX on Apple Silicon — for the MTP tensors, not the quant.

Not loadable by llama.cpp, vLLM, or Transformers (MLX safetensors either way). If you want vision on KAT under llama.cpp, you don't need this repo at all — see Alternatives.

What was done

KAT ships MoE experts per-expert (experts.{i}.{gate,up,down}_proj); mlx-vlm requires the fused experts.gate_up_proj / experts.down_proj layout. Fusing them is a pure re-packing — verified bit-exact by round-tripping random (layer, expert) samples back through mlx-vlm's own split.

The resulting tensor key set is identical to an oMLX oQ8e-mtp build of the stock base model: 2052 tensors — 1674 language_model.model, 42 language_model.mtp, 333 vision_tower, 3 lm_head.

Vision: honest framing

The vision tower was never trained with this checkpoint's language model. Kwaipilot stated plainly that they did no vision training. This is the base model's tower bolted to an RL-tuned coding LM.

That said — measured against the native Qwen3.6-35B-A3B under the same runtime and quant level, this build's vision behaves identically, including its failure modes:

So the two known weaknesses — a sparse, near-empty canvas with a few huge glyphs, and the Q→q substitution — are traits of the Qwen3.6 vision tower, not damage from the transplant.

In practice it does the job it exists for: reading code, terminal output and rendered UI off screenshots, and spotting layout defects. It is not a document-OCR model.

MTP (speculative decoding)

Restored and calibrated. Enable per-model in oMLX:

curl -X PUT http://127.0.0.1:8899/admin/api/models/KAT-Coder-V2.5-Dev-VL-oQ8e-mtp/settings \
  -H 'Content-Type: application/json' -d '{"mtp_enabled": true}'

Measured on an M5 Max (128 GB):

text-only  : accept 84-86%,  ~2.9-3.1 tok/cycle,  ~123 tok/s   (vs ~80 tok/s with MTP off)
with image : accept ~75%,    ~2.6 tok/cycle

Vision and MTP work on the same request — they are not mutually exclusive.

Speculative decoding with verification is token-identical to the target model, so MTP costs throughput if poorly calibrated, never quality.

Benchmarks

HumanEval, pass@1, thinking enabled, run locally under oMLX:

Re-measured 2026-08-04: 85.98% (141/164), 858 s. Same checkpoint, same settings — a 5-problem swing from the 146/164 above. Speculative decoding plus Metal batching numerics means the deterministic profile is not bit-reproducible, so treat any single pass@1 here as ±5 problems. The gap to the base (11 problems) still clears that noise; a 1–2 problem difference would not.

Other levels of this build

Same bf16 master, same verification, measured the same day:

Read that column as a tie. The between-level spread is no larger than the same-model run-to-run variance noted above, so 8, 6, and 4-bit are indistinguishable on HumanEval. Notably the MTP head did not degrade at 4-bit, which oMLX warns can happen — the imatrix was recomputed with the head present rather than reused from a pre-graft cache.

The 6.7-point gap is KAT's design, not quantization or transplant damage. KAT was RL-tuned to reason concisely; it emits 12.8× fewer reasoning characters and runs 6.6× faster. On the 12 problems this build missed and the base solved, the median reasoning length was 436 chars vs the base's 10,034 — it under-thought, it didn't mis-reason. This is exactly the tradeoff predicted in the community discussion before anyone had measured it.

Caveats on that 89.02%, in the interest of not overselling or underselling:

• ~3 of the 18 failures are not capability failures — 2 blew past the generation cap on runaway reasoning (one at 29k chars), and 1 produced a correct implementation that was graded wrong for returning a bare return line instead of the complete function. True capability is ~90–91%.
• This is a one-shot benchmark. In an agentic loop with test feedback, 3.0 s/problem buys ~6 attempts in the time the base takes 1.
• I did not run a matched control against a text-only KAT build at the same quant level, so this compares this build to the base — not this build to stock KAT. The mechanism (systematically shorter reasoning) argues strongly it is KAT being KAT; a weight transplant produces wrong answers, not shorter ones. But that is an argument, not a measurement.

Limitations

• oMLX only as published — because of the MTP head, not the quantization. Stock mlx-vlm 0.6.3 fails on the 42 language_model.mtp.* tensors. Strip them (and set text_config.mtp_num_hidden_layers = 0) and the remainder is a portable oQ8e checkpoint that loads in stock MLX apps — you lose speculative decoding, keep vision and the text model.
• Vision is untrained for this checkpoint. Good for screenshots of code/terminals/UI; unreliable for isolated large glyphs and occasionally for single characters in random strings.
• Set a max_tokens cap. The observed worst case was 29k characters of reasoning on a single problem. In an agent loop that stalls rather than errors.
• Quantized to 8-bit. Lower oQ levels of an MTP-preserved head can collapse draft acceptance — re-check the acceptance rate if you requantize downward.
• Chinese/English only, inherited from the base.

Reproducing

Fully scripted and documented — nothing here is hand-edited. Summary:

1. Fuse KAT's per-expert MoE tensors into the fused layout mlx-vlm expects (gate first, then up).
2. Copy the 333 model.visual.* and 19 mtp.* tensors from Qwen/Qwen3.6-35B-A3B (only 4 of its 26 shards are needed), set text_config.mtp_num_hidden_layers = 1.
3. Quantize with oMLX at oq_level=8, enhanced=true, preserve_mtp=true.

Use the official Qwen repo as the MTP source, not a re-upload: oMLX's donor validation requires a byte-identical tokenizer.json, and KAT's matches Qwen's exactly (sha256 5f9e4d49…) while third-party re-packs generally do not.

Note that oMLX's mtp_assistant_model_path donor route is broken for MoE donors at the time of writing — it copies the head's expert tensors through without the switch_mlp conversion, and the model then silently loads without vision. Use preserve_mtp with the tensors already in the bf16 source, as above.

Alternatives

• llama.cpp: you don't need a merged model. Any KAT GGUF plus unsloth/Qwen3.6-35B-A3B-GGUF's mmproj-BF16.gguf via --mmproj gives working vision.
• Transformers/vLLM: beyoru/KAT-Coder-V2.5-Dev-VL is the equivalent safetensors graft (vision only, no MTP).

Credits

• Kwaipilot — KAT-Coder-V2.5-Dev (technical report)
• Qwen — Qwen3.6-35B-A3B, source of the vision tower and MTP head
• oMLX — quantization and serving
• The KAT-Coder community discussions, where the vision and MTP transplants were first proposed and tested

Apache-2.0, inherited from both parents.