Qwen3.6-27B-DSpark-49k

Qwen3.6-27B-DSpark-49k

A DSpark draft head for Qwen3.6-27B, trained on a mixed agentic + general-chat corpus and vocabulary-truncated for serving. Drops into stock vLLM as a --speculative-config.

2.80× faster decoding than no speculation, 1.59× faster than the upstream DSpark head, and 1.36× faster than Qwen3.6's built-in MTP, at 4.66 tokens accepted per target pass, in 6.20 GiB instead of 8.20 GiB. Measured over 96 held-out prompts spanning agentic tool-use, coding and general chat, with generation length and sampling fixed so wall-clock is directly comparable.

Two builds of this head. This is the vocabulary-truncated build (49,167 draft tokens, 6.20 GiB) — prefer it for short-prompt and chat-shaped serving. The full-vocabulary build of the same trained weights is at abstract-extraordinary/Qwen3.6-27B-DSpark (8.20 GiB), and is the better choice for long-context agentic work. See Vocabulary truncation.

Usage

vllm serve unsloth/Qwen3.6-27B-NVFP4 \
  --speculative-config '{"method":"dspark",
                         "model":"abstract-extraordinary/Qwen3.6-27B-DSpark-49k",
                         "num_speculative_tokens":8}' \
  --trust-remote-code

Use num_speculative_tokens: 8. The head drafts blocks of exactly 8 tokens (block_size=8), so a smaller value leaves trained capacity unused and a larger one asks it to predict positions it has never seen.

Benchmarks

96 held-out prompts across 8 workload buckets. All five configurations ran in a single session against the same prompts, identical but for --speculative-config, each emitting exactly 300 greedy tokens (temperature=0, min_tokens=max_tokens, ignore_eos) so wall-clock is directly comparable. Target unsloth/Qwen3.6-27B-NVFP4, single GB10, concurrency 1. Every figure is a draft-weighted pool over the whole run, not a mean over prompts.

The upstream head was run at both num_speculative_tokens: 8 and 15 (its own block_size); 8 was faster (17.74 vs 17.07 tok/s) and is quoted here, so the baseline is shown at its stronger setting.

MTP shows a much higher acceptance percentage while being slower. The two are not in tension: MTP drafts 3 tokens per pass and this head drafts 8, so a higher hit rate on fewer guesses still yields fewer accepted tokens per step (3.25 vs 4.66). Tokens per step is the quantity that converts into wall-clock.

Acceptance by draft position

Acceptance decays multiplicatively: surviving to position 3 means winning four consecutive rejection tests. That compounding is what separates a trained head from the untrained upstream one — upstream starts at a respectable 62.4% at position 0 but halves roughly every two positions to 5.8% by position 7, while this head starts at 82.1% and still retains 22.4%. Note how little daylight there is between the two truncated and full-vocab curves: truncation costs very little acceptance in aggregate (see below).

Vocabulary truncation — read this before deploying

lm_head and markov_w2 are row-gathered to the 49,167 tokens the target actually emits (ranked over 35.8M tokens from 96,094 of this head's own generations; 99.44% coverage on a held-out split, 0.56% OOV against a ~3.3% break-even), shipped with a d2t offset map. markov_w1 stays full width — it is indexed by target ids, and gathering it would corrupt every sequential Markov step.

This is a post-hoc transform of trained weights, not a retrain. It wins wall-clock back because on a bandwidth-bound accelerator the draft head streams its whole weight set every decode step, and markov_w2 is re-read at every sequential Markov position. Overall it is +7.2% tok/s (26.28 → 28.18) for 2 GiB less memory.

The average hides a strong interaction with prompt length, and that interaction is the thing to deploy against. Paired per prompt against the full-vocab build:

Two mechanisms. Throughput: on long prompts prefill dominates wall-clock, so shrinking the draft head's per-step bandwidth barely registers; on short prompts decode dominates and the saving shows up directly. Acceptance: the keep-list was ranked on generations that are roughly 2:1 chat by token count, so agentic and code vocabulary — identifiers, tool-call syntax — falls outside the kept set more often.

If your workload is long-context agentic (multi-thousand-token prompts), use the full-vocab build instead — there truncation buys no measurable throughput while the acceptance cost is real. Truncation is a clear win for short-prompt and chat-shaped serving. If you want truncation for agentic traffic, re-rank the keep-list with agentic weighted up or raise K, and re-benchmark.

On the aggregate acceptance cost: two independent paired runs disagree on its size — one measured −0.163 tok/step (t = −2.47), the other −0.037 (t = −0.49). Treat the whole-corpus figure as "small and not reliably measurable"; the length-conditional table above reproduced cleanly in both runs and is the one to plan against.

Training

Assistant turns regenerated by the target model itself (temperature 0.7, top-p 0.95), so the head learns the distribution it will actually have to predict — not the distribution of whatever model produced the original traces.

14,707 conversations from seven sources: fresh nvidia/Open-SWE-Traces windows, three agentic trace sets, and three general-chat sets. 44.93M supervised tokens at 46.6% loss density (mean 3,055 per sample). 600 steps in 6.3 h on one node, lr 6e-4, 384 anchors, block_size=8; final loss 0.73 from ~1.05. Held-out offline accepted length was monotone across all 20 checkpoints and plateaued at steps 540–600, so the run converged rather than being truncated.

Held-out prompts were excluded from training by content fingerprint before harvesting, train/val shard offsets verified disjoint, and the publishers' own splits respected where they exist. All evaluation above uses only held-out samples.

Credits

Initialised from satgeze/Qwen3.6-27B-DSpark. DSpark architecture from DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation. Vocabulary truncation follows FR-Spec. Training data from NVIDIA and the community dataset authors listed above.