KAT-Coder-V2.5-Dev-REAP-50-bf16

KAT-Coder-V2.5-Dev · REAP-50 (pruned bf16)

The 50%-REAP-pruned Kwaipilot/KAT-Coder-V2.5-Dev in bf16 — the source checkpoint behind the NVFP4 and GGUF releases. Provided so others can produce their own quants (AWQ, EXL2, MLX, custom GGUF, …).

• 256 → 128 experts via REAP (Router-weighted Expert Activation Pruning), with a router-renormalization fix over the survivors.
• qwen3_5_moe hybrid (Gated-DeltaNet + attention + MoE). No MTP head (mtp_num_hidden_layers: 0); reload-verified with a real forward pass.
• Vision tower not stripped in this checkpoint — use Qwen3_5MoeForCausalLM for text-only.

Base-model quality (measured on the NVFP4A16 quant of this checkpoint, greedy, instruct): HumanEval+ ~90%, MBPP+ ~90%. These are single greedy draws; the accuracy harness has a measured run-to-run spread of 1.85–4.27 pp per task (2026-09-05), so read small differences as unresolved.

Architecture

graph TD
    Base["Kwaipilot/KAT-Coder-V2.5-Dev<br/>Qwen3.5 MoE - 256 experts - ~69 GB bf16"]
    subgraph Build ["Build - RTX 5070 Ti, SM120"]
        REAP["REAP expert prune 50%  256 -> 128 experts + router-renorm fix"]
        Strip["strip vision tower + 333 untrained tensors"]
        Quant["NVFP4A16 quantize (weight-only, data-free, 82 s)"]
    end
    subgraph HF ["Published formats"]
        A16["REAP-50-NVFP4A16 - 12.45 GiB (default, vLLM)"]
        W4A4["REAP-50-NVFP4-W4A4 (native FP4)"]
        GPTQ["REAP-50-NVFP4A16-GPTQ (null result, kept for transparency)"]
        GGUF["REAP-50-GGUF (Q4_K_M / Q5_K_M / Q6_K / Q8_0)"]
        BF16["REAP-50-bf16 (pruned source)"]
    end
    Bench["A16 single draws - HumanEval+ 89.0% - MBPP+ 90.5% - SWE-bench Verified 52.0%"]
    Base --> REAP --> Strip --> Quant --> A16
    Strip --> BF16
    BF16 -. re-quant .-> W4A4
    BF16 -. re-quant .-> GPTQ
    BF16 -. convert .-> GGUF
    A16 --> Bench

Releases built from this

• NVFP4A16 (vLLM / Blackwell): Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4A16
• NVFP4 W4A4: Ttimms/KAT-Coder-V2.5-Dev-REAP-50-NVFP4-W4A4
• GGUF: Ttimms/KAT-Coder-V2.5-Dev-REAP-50-GGUF

Pipeline

Full prune → quant → serve → evaluate pipeline: https://github.com/t-timms/kat-coder-nvfp4

License

Apache-2.0 (inherits from Kwaipilot/KAT-Coder-V2.5-Dev). Pruning via REAP (github.com/CerebrasResearch/reap, with a router-renormalization fix).