KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16-GPTQ

KAT-Coder-V2.5-Dev REAP-50 NVFP4A16 — GPTQ rounding variant

Published for completeness and independent verification, not as a recommended alternative. If you're choosing a checkpoint, use Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16 — that is the primary release, actively maintained, and this card exists specifically to document why this variant is not a better choice.

Architecture

graph TD
    Base["Kwaipilot/KAT-Coder-V2.5-Dev<br/>Qwen3.5 MoE - 256 experts - ~69 GB bf16"]
    subgraph Build ["Build - RTX 5070 Ti, SM120"]
        REAP["REAP expert prune 50%  256 -> 128 experts + router-renorm fix"]
        Strip["strip vision tower + 333 untrained tensors"]
        Quant["NVFP4A16 quantize (weight-only, data-free, 82 s)"]
    end
    subgraph HF ["Published formats"]
        A16["REAP-50-NVFP4A16 - 12.45 GiB (default, vLLM)"]
        W4A4["REAP-50-NVFP4-W4A4 (native FP4)"]
        GPTQ["REAP-50-NVFP4A16-GPTQ (null result, kept for transparency)"]
        GGUF["REAP-50-GGUF (Q4_K_M / Q5_K_M / Q6_K / Q8_0)"]
        BF16["REAP-50-bf16 (pruned source)"]
    end
    Bench["A16 single draws - HumanEval+ 89.0% - MBPP+ 90.5% - SWE-bench Verified 52.0%"]
    Base --> REAP --> Strip --> Quant --> A16
    Strip --> BF16
    BF16 -. re-quant .-> W4A4
    BF16 -. re-quant .-> GPTQ
    BF16 -. convert .-> GGUF
    A16 --> Bench

What this is

Same REAP-50%-pruned source checkpoint as the primary release, quantized to the identical NVFP4A16 scheme (weight-only, 4-bit) — but with GPTQ (Hessian-based, error-correcting) rounding instead of the primary release's plain round-to-nearest (RTN). Same size, same architecture, same serving kernel. The only variable changed is the quantization algorithm.

12.4512 GiB — exact byte-for-byte size match to the primary release.

Why this was tried

GPTQ is documented in current literature to generally outperform RTN for NVFP4 weight recovery. It was tested here specifically to see whether that general finding held for this checkpoint and closed any of the gap to the competitive bar (Devstral Small 2512, 56.4% SWE-bench Verified under the same mini-swe-agent bash-only scaffold).

Result: no measurable difference from the primary release

Paired McNemar test (not just comparing point estimates) against the primary release's own accuracy suite, same evaluation methodology:

Both comparisons are underpowered (well under the ~40-50 discordant pairs needed for adequate statistical power) — this is genuinely "no detectable difference," not confident evidence of true equivalence, and not evidence GPTQ is worse either. The run itself was clean: 5h23min, zero exceptions, zero fallback-to-RTN warnings across all 15,520 quantized modules.

Practical read: this specific checkpoint's RTN baseline appears to already be close to its own accuracy ceiling, leaving little room for a better rounding algorithm to show up. GPTQ's general advantage over RTN (well-supported in the literature for other models) doesn't disappear — it just isn't visible here at the precision this evaluation can resolve.

Full writeup, including the literature verification done before running this and the exact promotion criteria this didn't clear: ROADMAP.md in the primary release's GitHub repo.

Quantization details

License

Apache 2.0, inherited from the base model Kwaipilot/KAT-Coder-V2.5-Dev.