Qwen3.6-27B-DSpark

Qwen3.6-27B-DSpark

A DSpark draft head for Qwen3.6-27B, trained on a mixed agentic + general-chat corpus. Drops into stock vLLM as a --speculative-config.

2.62× faster decoding than no speculation, 1.48× faster than the upstream DSpark head, and 1.27× faster than Qwen3.6's built-in MTP, at 4.67 tokens accepted per target pass. Measured over 96 held-out prompts spanning agentic tool-use, coding and general chat, with generation length and sampling fixed so wall-clock is directly comparable.

Two builds of this head. This is the full-vocabulary build (8.20 GiB) — prefer it for long-context agentic work. A vocabulary-truncated build of the same trained weights is at abstract-extraordinary/Qwen3.6-27B-DSpark-49k (6.20 GiB, faster on short prompts, no throughput advantage on long ones). See Vocabulary variants.

Usage

vllm serve unsloth/Qwen3.6-27B-NVFP4 \
  --speculative-config '{"method":"dspark",
                         "model":"abstract-extraordinary/Qwen3.6-27B-DSpark",
                         "num_speculative_tokens":8}' \
  --trust-remote-code

Use num_speculative_tokens: 8. The head drafts blocks of exactly 8 tokens (block_size=8), so a smaller value leaves trained capacity unused and a larger one asks it to predict positions it has never seen.

Benchmarks

96 held-out prompts across 8 workload buckets. All five configurations ran in a single session against the same prompts, identical but for --speculative-config, each emitting exactly 300 greedy tokens (temperature=0, min_tokens=max_tokens, ignore_eos) so wall-clock is directly comparable. Target unsloth/Qwen3.6-27B-NVFP4, single GB10, concurrency 1. Every figure is a draft-weighted pool over the whole run, not a mean over prompts.

The upstream head was run at both num_speculative_tokens: 8 and 15 (its own block_size); 8 was faster (17.74 vs 17.07 tok/s) and is quoted here, so the baseline is shown at its stronger setting.

MTP shows a much higher acceptance percentage while being slower. The two are not in tension: MTP drafts 3 tokens per pass and this head drafts 8, so a higher hit rate on fewer guesses still yields fewer accepted tokens per step (3.25 vs 4.67). Tokens per step is the quantity that converts into wall-clock.

Acceptance by draft position

Acceptance decays multiplicatively: surviving to position 3 means winning four consecutive rejection tests. That compounding is why a few points at position 0 turn into a large difference in tokens per step, and it is what separates this head from the untrained upstream one. Upstream starts at a respectable 62.4% at position 0 but halves roughly every two positions to 5.8% by position 7; this head starts at 82.7% and still retains 22.8%.

Vocabulary variants

Both are the same trained weights. The -49k build row-gathers lm_head and markov_w2 to the tokens the target actually emits — a post-hoc transform, not a retrain. It is faster per decode step because the draft head streams its whole weight set every step and a smaller head streams less. That saving is large on short prompts and disappears on long ones, where prefill dominates wall-clock, while a small acceptance cost remains. If your prompts run to thousands of tokens, use this full-vocab build.

Training

Assistant turns regenerated by the target model itself (temperature 0.7, top-p 0.95), so the head learns the distribution it will actually have to predict — not the distribution of whatever model produced the original traces.

14,707 conversations from seven sources: fresh nvidia/Open-SWE-Traces windows, three agentic trace sets, and three general-chat sets. 44.93M supervised tokens at 46.6% loss density (mean 3,055 per sample). 600 steps in 6.3 h on one node, lr 6e-4, 384 anchors, block_size=8; final loss 0.73 from ~1.05. Held-out offline accepted length was monotone across all 20 checkpoints and plateaued at steps 540–600, so the run converged rather than being truncated.

Held-out prompts were excluded from training by content fingerprint before harvesting, train/val shard offsets verified disjoint, and the publishers' own splits respected where they exist. All evaluation above uses only held-out samples.

Credits

Initialised from satgeze/Qwen3.6-27B-DSpark. DSpark architecture from DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation. Training data from NVIDIA and the community dataset authors listed above.