Qwen3-4B-speculator-b8-dpard.dspark

Qwen3-4B DSpark Speculator (B8, D-PARD)

This repository contains a DSpark speculative decoding draft model trained for Qwen/Qwen3-4B. It proposes up to eight tokens per decoding iteration and was trained with the D-PARD position-weighting objective.

This is a research checkpoint intended for use with vllm-project/speculators and vLLM. It is not a standalone language model.

Model details

• Architecture: DSpark draft model
• Verifier: Qwen/Qwen3-4B
• Speculative block size: 8
• Draft transformer layers: 5
• Verifier hidden-state layers: 1, 9, 17, 25, 33
• Markov head: vanilla, rank 256
• Confidence head: enabled with Markov features
• Default proposal method: greedy
• Precision: bfloat16
• Draft vocabulary size: 151,936
• Parameters file size: 1.23 GB

Training

The model was trained for one epoch from git commit 25f4e15859e58d6b1826083ef052f076f4480574.

• Global steps: 244,909
• World size: 4
• Sequence length: 8,192
• Optimizer: AdamW
• Learning rate: 6e-4
• Weight decay: 0
• Scheduler: cosine with 4% warmup
• Seed: 42
• Loss: Rényi-half
• Position weighting: D-PARD, alpha 0.5
• speculators: 0.7.0.dev150
• Training vLLM: 0.26.0
• Transformers: 5.12.0
• PyTorch: 2.11.0

The full invocation is included in train_command.txt.

Validation results

Speculator benchmark results

The checkpoint was evaluated with the D-PARD evaluation implementation on RedHatAI/speculator_benchmarks, using temperature 0, maximum generation length 128, maximum concurrency 128, and up to 200 requests per category.

The weighted overall result is computed as 1 + total accepted speculative tokens / total draft iterations.

Under the same evaluation settings, RedHatAI/Qwen3-4B-speculator.dspark achieved a weighted acceptance length of 4.7543. Results were very close and varied by category.

Usage

Use this repository as the speculative model for Qwen/Qwen3-4B with a version of vLLM and speculators that supports DSpark. Keep the verifier model and tokenizer set to Qwen/Qwen3-4B; the configured speculative-token count is 8.

Limitations

• Acceptance and speedup depend on prompts, hardware, batch size, vLLM version, and serving configuration.
• This checkpoint inherits the behavior, limitations, and license considerations of its verifier model.
• Acceptance length alone does not determine end-to-end throughput.