Downloads · 30 days
556
1% of all-time downloads
Qwen3-8B-speculators.peagle-qwen3arch-ckpt4
inference-optimization/Qwen3-8B-speculators.peagle-qwen3arch-ckpt4
Qwen3-8B-speculators.peagle-qwen3arch-ckpt4 is a machine learning model from inference-optimization. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for speculators. The card lists the license as apache-2.0.
P-EAGLE (Parallel Eagle) speculative decoding drafter for Qwen/Qwen3-8B using the Qwen3 draft architecture (--draft-arch qwen3).
All-time downloads
59.2K
Public
Parameters
1.6B
3.2 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors3.2 GB · 100%
Parameter types
How the weights are stored.
BF161.6B · 100%
Base models
Model card
From the Hugging Face model README
Qwen3-8B P-EAGLE Drafter (Qwen3 Architecture)
P-EAGLE (Parallel Eagle) speculative decoding drafter for Qwen/Qwen3-8B using the Qwen3 draft architecture (--draft-arch qwen3).
The Qwen3 architecture uses additional q_norm/k_norm layers in attention, which stabilize training at higher learning rates compared to the default Llama draft architecture. See speculators#563 for the RFC and experimental results.
Benchmark Results
Evaluated on RedHatAI/speculator_benchmarks (9 subsets) via vLLM throughput mode.
Acceptance lengths
| Subset | k=5 | k=7 |
|---|---|---|
| HumanEval | 3.51 | 3.73 |
| Math Reasoning | 3.73 | 4.04 |
| QA | 2.81 | 3.01 |
| Question | 2.99 | 3.27 |
| RAG | 3.06 | 2.98 |
| Summarization | 2.72 | 2.55 |
| Tool Call | 2.71 | 2.97 |
| Translation | 2.80 | 2.81 |
| Writing | 3.00 | 3.37 |
| Average | 3.04 | 3.19 |
Comparison vs Llama-arch baseline
Format: delta vs Llama-arch P-EAGLE.
| Subset | k=5 | k=7 |
|---|---|---|
| HumanEval | +0.50 | +0.23 |
| Math Reasoning | +0.46 | +0.22 |
| QA | +0.41 | +0.19 |
| Question | +0.36 | +0.24 |
| RAG | +0.57 | +0.05 |
| Summarization | +0.69 | +0.07 |
| Tool Call | +0.26 | +0.10 |
| Translation | +0.54 | -0.07 |
| Writing | +0.37 | +0.31 |
| Average | +0.46 (+18%) | +0.15 (+5%) |
Qwen3-arch outperforms Llama-arch on all 9 subsets at k=5 and 8/9 at k=7.
Training
| Parameter | Value |
|---|---|
| Target model | Qwen/Qwen3-8B |
| Draft architecture | Qwen3 (--draft-arch qwen3) |
| Learning rate | 6e-4 |
| Epochs | 5 |
| Draft vocab size | 32000 |
| Sequence length | 8192 |
| Num layers | 4 |
| Num depths | 7 |
| Down-sample ratio | 0.6 (min 0.2) |
| Scheduler | Cosine |
| Hardware | 2x H200 |
| Training library | speculators |
Usage
Requires vLLM with Qwen3 Eagle3/P-EAGLE support (vllm#43132) and the architecture resolution fix.
vllm serve Qwen/Qwen3-8B \
--speculative-config '{
"model": "inference-optimization/Qwen3-8B-speculators.peagle-qwen3arch-ckpt4",
"num_speculative_tokens": 7,
"method": "eagle3",
"parallel_drafting": true
}'
Related
- speculators#563 — RFC: Support Qwen3 base architecture for Eagle3 & P-EAGLE
- Llama-arch P-EAGLE baseline — Same setup with Llama draft architecture
- Eagle3 Qwen3-arch drafter — Eagle3 (sequential) variant
- Library
- speculators
- License
- apache-2.0
- Created
- Jun 16, 2026
- Updated
- Jun 16, 2026