kimi-k2.7-coder-eagle3.1-mla

<!-- Draft: review the runtime requirements and benchmark wording before publishing. -->

Kimi-K2.7 Coder Eagle3.1 MLA

EAGLE3 draft model for speculative decoding with Kimi-K2.7-Code.

Continued training from kimi-k2.6-eagle3.1-mla.

Co-authored by KVCache.AI and LightSeek.

This checkpoint uses MLA and a one-layer EAGLE3.1 draft architecture with fc_norm and norm_output.

Features

• EAGLE3.1 + MLA: One-layer draft model using auxiliary hidden states from target layers 1, 29, and 57.
• fc_norm: Per-chunk RMSNorm on auxiliary hidden states before FC projection.
• norm_output: Uses post-norm hidden states as auxiliary output.
• Long-context configuration: YaRN RoPE configuration with a maximum position length of 262,144 tokens.

Benchmark Results

Target model: moonshotai/Kimi-K2.7-Code

Draft model: lightseekorg/kimi-k2.7-coder-eagle3.1-mla

3-token draft

The run used SGLang 0.5.14, TP=8, EP=1, BF16, topk=1, num_steps=3, num_draft_tokens=4.

Usage with vLLM

https://recipes.vllm.ai/moonshotai/Kimi-K2.7-Code

Usage with SGLang

<!-- TODO: Replace the target model path and verify the minimum supported SGLang version before publishing. -->

python3 -m sglang.launch_server \
  --model-path /path/to/Kimi-K2.7-Code \
  --tp 8 --ep-size 1 --trust-remote-code \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path lightseekorg/kimi-k2.7-coder-eagle3.1-mla \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --decode-attention-backend flashinfer \
  --speculative-draft-attention-backend flashinfer \
  --mem-fraction-static 0.75 \
  --page-size 1 \
  --max-running-requests 8 \
  --dtype bfloat16 \
  --host 0.0.0.0 \
  --port 30000 \
  --reasoning-parser kimi_k2 \
  --tool-call-parser kimi_k2 \
  --cuda-graph-backend-decode full \
  --cuda-graph-max-bs-decode 1 \
  --cuda-graph-bs-decode 1 \