KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B

KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B

A Mixture-of-Experts DFlash2 drafter for Kwaipilot/KAT-Coder-V2.5-Dev.

Modern MoE target models are well suited to expert offloading on memory-constrained systems and large-scale expert parallelism at datacenter scale. Applying the same sparse structure to speculative drafters is therefore a natural direction. Strong precedents include DeepSeek-V3 MTP and DeepSeek-V4-Flash-0731 DSpark.

This release explores the same direction with a DFlash2 drafter: approximately 2.6B total parameters with approximately 0.3B activated per token.

Results

Accepted length is calculated from the raw server counters:

accepted_length = 1 + accepted_tokens / draft_calls

Architecture

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with KAT-Coder or a compatible quantized version:

vllm serve Kwaipilot/KAT-Coder-V2.5-Dev \
  --tensor-parallel-size 1 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --language-model-only \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/KAT-Coder-V2.5-Dev-DFlash2-2.6B-A0.3B",
    "num_speculative_tokens": 7
  }'

This checkpoint currently requires a vLLM build containing both vLLM PR #54154, which adds the sample_from_anchor=true DFlash query layout, and vLLM PR #54475, which adds MoE DFlash2.

Checkpoint

The BF16 checkpoint contains 63 tensors and 2,638,551,808 parameters.

Future work

• Adaptive verification: explore confidence-head-driven adaptive draft budgets for high-concurrency serving.

Acknowledgements

This work builds on Kwaipilot/KAT-Coder-V2.5-Dev, vllm-project/speculators, and vllm-project/vllm.

License

Apache-2.0.