Qwen3.6-27B-DSpark-FR

Qwen3.6-27B-DSpark-FR

A DSpark speculative-decoding drafter for Qwen/Qwen3.6-27B, trained on French on-policy data. Warm-started from z-lab's public DFlash drafter, converted to the DSpark format (rank-256 Markov head + confidence head), then fine-tuned.

To our knowledge this is the first DSpark drafter for Qwen3.6-27B adapted to French, and the first with a measured comparison showing it beats both DFlash and native Multi-Token Prediction (MTP).

Motivation

The public DFlash drafter plateaued at 8–20% acceptance on French editorial prompts (vs 23–50% on English math/code workloads). Diagnosis: domain mismatch. This model fixes that deficit via fine-tuning on French on-policy data. The technique itself is language-agnostic — the recipe applies to any target domain.

Measured results (DGX Spark GB10, NVFP4-FR body, 150 held-out FR prompts)

• +16.9% relative acceptance and +14.4% end-to-end throughput vs DFlash.
• +50% throughput vs native MTP on the identical body: although MTP has higher per-token acceptance, DSpark drafts deeper (n=8 vs k=3) and emits more useful tokens per verification pass.
• Acceptance is invariant (23.4–23.8%) across four quantizations of the target body → robust to body changes.
• Works under multi-node tensor parallelism (TP=2) with no acceptance loss (23.3% across two DGX Sparks over QSFP) → 28.79 tok/s @ c1. First known validation of DSpark in inter-node tensor parallelism.

Speculative decoding is lossless by design — rejection sampling reproduces the target model's distribution exactly.

⚠️ Caveat on hybrid Mamba/Gated-DeltaNet targets (verified 2026-07-14). A vLLM implementation bug (#43559) corrupts the recurrent state when prefix caching and speculative decoding are both enabled on hybrid bodies such as Qwen3.6 (the SSM/GDN state is advanced over all drafted tokens but not rolled back to the accepted position). This silently degrades tool-calling / structured output — measured 43 vs 88/100 on tool-eval-bench (Parameter Precision and Multi-Step Chains collapse to 0%), while free-form prose is unaffected. Fix: vLLM PR #48361, or run with --no-enable-prefix-caching. Pure speculative decoding (no prefix cache) is lossless as expected.

Geometry

• Architecture: DSparkDraftModel (5-layer DFlash backbone + rank-256 Markov head + confidence head).
• block_size 16, served at num_speculative_tokens=8.
• ~1.9B parameters (embeddings/lm_head shared with the target).

Usage (vLLM)

Used with the target body via --speculative-config:

vllm serve <Qwen3.6-27B-body> \
  --speculative-config '{"method":"dspark","model":"pablohassan/Qwen3.6-27B-DSpark-FR","num_speculative_tokens":8}'

Recommended with Qwen3.6-27B-NVFP4-FR. Compatible with other quantizations of the same body (acceptance is stable).

Requires a recent vLLM main build with DSpark support.

Training

• Framework: speculators (vllm-project), warm-start DFlash → DSpark.
• Corpus: 3,000 prompts (2,550 FR + 450 EN), responses regenerated on-policy by the target model, hidden states extracted via extract_hidden_states.
• Recipe: 6 epochs, lr 6e-4 → 4e-4, γ=6.0, losses CE 0.1 / TV 0.9.

Lineage & license

• Base: z-lab/Qwen3.6-27B-DFlash (MIT).
• License: MIT (inherited).

Links

• Recommended target body: https://huggingface.co/pablohassan/Qwen3.6-27B-NVFP4-FR
• Code, recipe and technical report: https://github.com/Pablohassan/qwen3.6-27b-fr-nvfp4-dspark