qwen3.6-27B-domino

Qwen3.6-27B-Domino

Paper | GitHub

This is a block-size-16 Domino draft model for the target model Qwen/Qwen3.6-27B. It is intended for speculative decoding with the Domino-enabled SGLang branch.

Quick Start

Install the matching SGLang branch:

uv pip install "git+https://github.com/jianuo-huang/sglang.git@feat/domino-tensor-parallel#subdirectory=python"

Launch the regular b16 configuration:

python -m sglang.launch_server \
  --model-path Qwen/Qwen3.6-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path huang2020/Qwen3.6-27B-Domino \
  --speculative-dflash-block-size 16 \
  --speculative-num-draft-tokens 16 \
  --tp-size 2 \
  --attention-backend flashinfer \
  --mamba-scheduler-strategy extra_buffer \
  --trust-remote-code

For b8, set --speculative-num-draft-tokens 8 while keeping --speculative-dflash-block-size 16.

SGLang implementation: PR #32018.

Benchmark

Methods. AR is target-only decoding. MTP-S3/S7/S15 use the built-in Qwen3.6 MTP heads with 3/7/15 steps, 4/8/16 draft tokens, and top-k 1. DFlash uses the official z-lab/Qwen3.6-27B-DFlash checkpoint at revision 0919688.

For DFlash and Domino, b16 is the regular block-16 run and the target verifies all 16 positions. b8 keeps the same block-16 draft backbone but the target verifies only the first 8 positions. The draft backbone itself is not shortened.

Setting. Qwen/Qwen3.6-27B, TP2/BF16 on 2×A100 80GB, FlashInfer, O4096, thinking enabled, greedy sampling (temperature=0, top_p=1, top_k=1), C1/C8/C32, and three fresh-server repeats per cell. Workloads are GSM8K-128, MATH500-128, HumanEval-164, MBPP-128, MT-Bench-80, and Alpaca-128.

Each bar is mean output tok/s over three runs. The black outline marks the fastest speculative configuration for each workload.

Throughput and speedup

Each cell is output tok/s (speedup versus AR). Bold marks the fastest speculative configuration in each row.

Concurrency 1

Concurrency 8

Concurrency 32

Macro speedup vs AR

Arithmetic mean of the per-workload TPS ratios; Overall is the arithmetic mean of all 18 workload-by-concurrency ratios.

Accept length

Mean output tokens per target verification step, including the target bonus token. The maxima are 4/8/16 for MTP-S3/S7/S15, 8 for b8, and 16 for b16. Bold marks the highest value within the directly comparable max-8 and max-16 groups.

Concurrency 1

Concurrency 8

Concurrency 32

All 144 displayed method/workload/concurrency cells completed three measured runs.

These are observed serving results. BF16 greedy trajectories can differ across methods, so the TPS differences are not pure kernel-attribution claims.

Citation

@article{huang2026domino,
  title={Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding},
  author={Huang, Jianuo and Zhang, Yaojie and Zhang, Qituan and Lin, Hao and Xu, Hanlin and Zhang, Linfeng},
  journal={arXiv preprint arXiv:2605.29707},
  year={2026}
}

Acknowledgements

Domino builds on DFlash and its block-diffusion drafting formulation.