Downloads · 30 days
12.5K
89% of all-time downloads
kimi-k3-dflash2
lightseekorg/kimi-k3-dflash2
kimi-k3-dflash2 is a machine learning model from lightseekorg. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
Draft model for speculative decoding on moonshotai/Kimi-K3, served with TokenSpeed (--speculative-algorithm DFLASH).
All-time downloads
14.1K
Public
Parameters
3.7B
23.1 GB on disk
Likes
5
Public
Repo makeup
Click a slice to open those files.
.safetensors7.4 GB · 100%
At a glance
- License
- other
- Model type
- qwen3
- Access
- Public
- Created
- Aug 23, 2026
- Updated
- Sep 2, 2026
- SHA
- e77935fb
Spaces
Model card
From the Hugging Face model README
Kimi-K3 DFlash2 draft
Draft model for speculative decoding on moonshotai/Kimi-K3, served with
TokenSpeed (--speculative-algorithm DFLASH).
Architecture
5-layer, lookahead-8, MLA + SWA
DFlash2DraftModel, 5 layers, hidden size 7168, bfloat16block_size: 8(lookahead-8),attention_mode: mlatarget_layer_ids: [19, 37, 66, 78, 90]- five auxiliary hidden-state tapslayer_types: 4xsliding_attention(window 4096) + 1xfull_attention- yarn RoPE,
rope_theta: 50000.0,factor: 32.0 - 3.69 B parameters, 7.39 GB of weights
Benchmarks
Acceptance length acc_len = 1 + accepted/steps at 7 speculative proposals,
served with TokenSpeed DFLASH on two GB300 nodes (8 GPUs, TP8), MLA target and
drafter. Target is moonshotai/Kimi-K3 with the serving flags shown at the bottom of this card.
Production sampling only: temperature=1.0, top_p=0.95, reasoning effort
max, no top_k.
Ten-benchmark matrix
| benchmark | acc_len | proposal acceptance | prompts | requests | truncated | mean out tok |
|---|---|---|---|---|---|---|
| GSM8K | 5.901 | 70.0% | 1319 | 1319 | 0.0% | 254 |
| HumanEval | 5.080 | 58.3% | 164 | 164 | 0.6% | 989 |
| MATH-500 | 4.636 | 51.9% | 500 | 500 | 0.8% | 803 |
| SPEED-Bench coding | 4.431 | 49.0% | 80 | 89 | 1.1% | 1341 |
| SPEED-Bench RAG | 3.958 | 42.3% | 80 | 91 | 1.1% | 691 |
| SPEED-Bench multilingual | 3.849 | 40.7% | 80 | 80 | 1.2% | 655 |
| MT-Bench | 3.585 | 36.9% | 80 | 160 | 9.4% | 1759 |
| SPEED-Bench QA | 3.213 | 31.6% | 80 | 80 | 3.8% | 1018 |
| AIME 2026 | 2.785 | 25.5% | 30 | 30 | 13.3% | 6091 |
| SPEED-Bench writing | 2.780 | 25.4% | 80 | 84 | 64.3% | 3588 |
Long context
RULER multi-key needle retrieval over natural-text paragraphs, with only the
input length varying. retrieval is exact match on the needle value, recorded
so that acceptance cannot be credited to a model that has stopped reading its
context.
| bucket | input tokens | acc_len | proposal acceptance | retrieval |
|---|---|---|---|---|
| 8k | 8203 | 5.666 | 66.7% | 1.000 |
| 32k | 32596 | 4.973 | 56.8% | 1.000 |
| 64k | 65113 | 4.805 | 54.4% | 1.000 |
| 128k | 130151 | 4.854 | 55.1% | 1.000 |
Serving
python3 -m tokenspeed.cli serve <Kimi-K3> \
--speculative-algorithm DFLASH \
--speculative-draft-model-path <this repo> \
--speculative-num-steps 7 --speculative-eagle-topk 1 \
--speculative-num-draft-tokens 8 \
--speculative-draft-model-quantization unquant \
--attention-backend mla --drafter-attention-backend mla \
--sampling-backend flashinfer --trust-remote-code \
--tensor-parallel-size 8 --moe-backend flashinfer_trtllm \
--max-model-len 32768 --gpu-memory-utilization 0.90 \
--max-num-seqs 4 --max-cudagraph-capture-size 4 \
--cudagraph-capture-sizes 1 2 3 4 \
--disable-prefill-graph --disable-kvstore
At startup the engine reports the auxiliary taps it captured; confirm they match
(19, 37, 66, 78, 90).
- Type
- qwen3
- License
- other
- Created
- Aug 23, 2026
- Updated
- Sep 2, 2026