Qwen3-4B-speculator-b16.dspark

Qwen3-4B DSpark Speculator (Block Size 16)

This repository contains a DSpark speculative decoding draft model trained for Qwen/Qwen3-4B. It proposes up to 16 tokens per decoding iteration and is intended for use with the vllm-project/speculators implementation and vLLM.

This is an experimental baseline checkpoint. It was trained for one epoch with the standard cross-entropy and total-variation objective; it was not trained with D-PARD.

Model details

• Architecture: DSpark draft model
• Verifier: Qwen/Qwen3-4B
• Speculative block size: 16
• Draft transformer layers: 5
• Verifier hidden-state layers: 1, 9, 17, 25, 33
• Markov head: vanilla, rank 256
• Confidence head: enabled with Markov features
• Default proposal method: greedy
• Precision: bfloat16
• Draft vocabulary size: 151,936
• Parameters file size: 1.23 GB

Training

The model was trained from the speculators main branch at commit ae96c74c0771ca4fb920df1279f41f60e54db2cc.

• Epochs: 1
• Global steps: 244,909
• World size: 4
• Sequence length: 8,192
• Optimizer: AdamW
• Learning rate: 6e-4
• Weight decay: 0
• Scheduler: cosine with 4% warmup
• Seed: 42
• Loss: 0.1 * CE + 0.9 * TV
• Position weighting: fixed exponential decay, gamma 4.0
• speculators: 0.7.0.dev150
• Training vLLM: 0.26.0
• Transformers: 5.12.0
• PyTorch: 2.11.0

The complete invocation is included in train_command.txt.

Validation results

Speculator benchmark results

The checkpoint was evaluated with the D-PARD evaluation implementation on RedHatAI/speculator_benchmarks, with temperature 0, maximum generation length 128, maximum concurrency 128, and up to 200 requests per category.

The weighted overall result is computed as 1 + total accepted speculative tokens / total draft iterations.

For context, under the same dataset and sampling settings, RedHatAI/Qwen3-4B-speculator.dspark using 8 speculative tokens achieved a weighted acceptance length of 4.7543. These numbers are not a strict architecture-controlled comparison because the models use different block sizes (16 versus 8).

Usage

Use this repository as the speculative model for Qwen/Qwen3-4B through a version of vLLM and speculators that supports DSpark. The verifier model and tokenizer must remain Qwen/Qwen3-4B, and the configured speculative-token count is 16.

Limitations

• This is a research checkpoint for speculative decoding, not a standalone language model.
• It inherits the behavior, limitations, and license considerations of the verifier model.
• Acceptance and speedup depend on prompts, hardware, batch size, vLLM version, and serving configuration.
• A larger speculative block does not guarantee higher average acceptance or end-to-end throughput.