Downloads · 30 days
39
18% of all-time downloads
Qwen3-0.6B-DSpark
yosefw/Qwen3-0.6B-DSpark
Qwen3-0.6B-DSpark is a machine learning model from yosefw. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
A DSpark draft model for speculative decoding with Qwen/Qwen3-0.6B as the verifier, trained with speculators. The drafter proposes 4 tokens at a time and the verifier checks them in one forward pass, so output is iden…
All-time downloads
211
Public
Parameters
317M
4.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors635 MB · 71%
Parameter types
How the weights are stored.
BF16317M · 100%
Base models
Model card
From the Hugging Face model README
Qwen3-0.6B-DSpark
A DSpark draft model for speculative decoding with Qwen/Qwen3-0.6B as the verifier, trained with speculators. The drafter proposes 4 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup.
Training code: rasyosef/train-dspark-draft-models.
Trained on 1,600 samples as a pipeline demonstration, not a deployment-ready drafter.
Usage
vLLM loads the verifier automatically from the config — don't pass it separately.
vllm serve yosefw/Qwen3-0.6B-DSpark --port 8000 --gpu-memory-utilization 0.75
Then query the OpenAI-compatible endpoint at http://localhost:8000/v1.
Details
5 Qwen3 layers (hidden size 1024, sliding-window attention), ~0.3B params, bfloat16. Block size 4, draft vocabulary reduced to 32,000, aux hidden-state layers 2/14/25, confidence head with Markov (rank 256).
Trained for 5 epochs at lr 3e-4 on 1,600 Magpie prompts regenerated by the verifier itself, with a {"ce": 0.1, "tv": 0.9} loss. Online mode: a vLLM server exported hidden states per batch during training. 2× T4, 8 h 54 min. speculators 0.8.0.dev197, vLLM 0.27.1.
Evaluation
evaluate.py throughput on 2× T4 across the nine RedHatAI/speculator_benchmarks subsets:
| subset | acceptance_length | pos_0 | pos_1 | pos_2 | pos_3 |
|---|---|---|---|---|---|
| math_reasoning | 1.9702 | 0.2673 | 0.1351 | 0.0647 | 0.0281 |
| HumanEval | 1.8709 | 0.2504 | 0.1168 | 0.0501 | 0.0204 |
| writing | 1.8068 | 0.2074 | 0.0953 | 0.0435 | 0.0190 |
| qa | 1.8011 | 0.2681 | 0.1258 | 0.0396 | 0.0188 |
| question | 1.7903 | 0.2753 | 0.1265 | 0.0543 | 0.0217 |
| tool_call | 1.7225 | 0.2036 | 0.0868 | 0.0384 | 0.0152 |
| rag | 1.6370 | 0.2114 | 0.0898 | 0.0390 | 0.0158 |
| summarization | 1.4854 | 0.1568 | 0.0569 | 0.0224 | 0.0090 |
| translation | 1.4356 | 0.1433 | 0.0519 | 0.0182 | 0.0063 |
Weighted across all subsets: 1.806 (435,398 accepted tokens over 539,885 drafts). Acceptance is highest where the verifier's next token is most predictable — code and math — and lowest on translation and summarization.
Limitations
Trained on only 1,600 samples, so acceptance would improve considerably with more data. Works only with Qwen/Qwen3-0.6B and is not usable as a standalone model. Real-world speedup depends on your traffic mix, and because verification is lossless, the verifier's own behavior and biases carry through unchanged.
Related
yosefw/Qwen3-0.6B-DSpark-v2 — same setup trained in offline mode on 1,200 samples.
License
Apache-2.0, matching both speculators and the verifier.
- Library
- transformers
- License
- apache-2.0
- Created
- Aug 23, 2026
- Updated
- Aug 27, 2026