Qwen3.8-27B-DSpark-vLLM

Qwen3.8-27B-DSpark — vLLM-ready

This is RadixArk/Qwen3.8-27B-DSpark with one line changed in config.json so that vLLM loads it.

The weights are byte-identical to the original. All credit for the drafter goes to RadixArk — this repo exists only so the checkpoint works with vLLM without a manual edit.

What was changed

- "architectures": ["DSparkDraftModel"]
+ "architectures": ["Qwen3DSparkModel"]

That's it.

Why it was needed

vLLM already supports this model natively — vllm/model_executor/models/qwen3_dspark.py plus a DSpark speculator have been in main for a while. But the registry has two DSpark entries:

"DSparkDraftModel": ("vllm.models.deepseek_v4", "DSparkDeepseekV4ForCausalLM"),
"Qwen3DSparkModel": ("qwen3_dspark", "Qwen3DSparkForCausalLM"),

The original checkpoint declares DSparkDraftModel, which routes to the DeepSeek-V4 class. vllm/config/speculative.py then rewrites model_type to deepseek_v4, and loading fails. Declaring Qwen3DSparkModel selects the Qwen3 implementation instead.

Every other field the original ships — dflash_config (target_layer_ids, mask_token_id, projector_type), markov_rank, enable_confidence_head, confidence_head_with_markov, layer_types, head_dim — is already exactly what Qwen3DSparkForCausalLM expects.

Usage

vllm serve /path/to/Qwen3.8-27B-GPTQ-Int4 \
  --dtype bfloat16 \
  --max-model-len 16384 \
  --max-num-seqs 16 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --block-size 64 \
  --trust-remote-code \
  --speculative-config '{
    "method": "dspark",
    "model": "/path/to/Qwen3.8-27B-DSpark-vLLM",
    "num_speculative_tokens": 7,
    "draft_sample_method": "probabilistic"
  }'

Measured results

Intel Arc Pro B70 32 GB, single card, vLLM 0.26.1rc1.dev799 XPU backend, V2 model runner. Target: Qwen3.8-27B GPTQ-Int4 (W4A16). Benchmarked with llama-benchy --pp 4096.

Per-position acceptance at k=7 (probabilistic): 0.76, 0.45, 0.31, 0.14, 0.10, 0.02, 0.00

Tuning notes

Use "draft_sample_method": "probabilistic". It measured ~23% faster than greedy on otherwise identical settings (52 vs 42 t/s). Greedy exact-match is a much harsher acceptance criterion than the sampling the published figures used.

Do not shrink the block, even though the acceptance curve says you should. Positions 6 and 7 accept ~2% and 0%, so trimming looks obviously correct. It isn't — k=4 and k=6 both measured worse than k=7. Per-step overhead dominates per-drafted-token cost, so amortising over more accepted tokens wins. Measure before trimming.

enable_adaptive_verification does not work with Qwen3.8. Its GDN linear-attention layers use GDNAttentionBackend, which vLLM rejects at startup:

ValueError: Adaptive verification trims verification requests on device, which
the GDNAttentionBackend attention backend does not support.

Architectural, not a misconfiguration — so the confidence head in this checkpoint currently goes unused under vLLM.

Acceptance is strongly workload-dependent

Measured in a standalone harness against the FP8 target, greedy exact-match:

This matches what the SGLang team reported (≈5.2 on gsm8k vs ≈2.9 on poetry). A single benchmark figure for this drafter is close to meaningless — reasoning-heavy traffic does substantially better than a generic tg32 benchmark suggests.

Target quantisation

The drafter reads target hidden states at layers 4/16/28/40/52. Those are bf16 activations regardless of how the target's weights are stored, so FP8, GPTQ-int4, AWQ and MXFP4 targets should all load.

Acceptance is a separate question: this drafter was trained against Qwen3.8-27B-FP8, and heavier quantisation perturbs the very hidden states it learned to read. GPTQ-int4 measured 2.45–2.79 here, but that isn't a like-for-like comparison with the published 3.39 (different sampling method and workloads). MXFP4 and AWQ are untested.

Limitations

• Verified on Intel Arc / XPU only. Nothing in the change is XPU-specific and CUDA should behave the same or better, but this is untested.
• Qwen3.8-27B only. The drafter's fc expects 5 × 5120 and taps layer indices up to 52, so other sizes in the family will not load.
• The FP8 target does not fit a single 32 GB card alongside the 2.7 GB drafter — no room is left for KV cache. Int4 leaves comfortable headroom.

Credit

All model weights and the DSpark method belong to RadixArk and the DSpark / SpecForge authors. This repo contributes a one-line config change and benchmark data on Intel Arc.