Downloads · 30 days
62
13% of all-time downloads
qwen3-eagle
ichsanlook/qwen3-eagle
qwen3-eagle is a machine learning model from ichsanlook. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
EAGLE-3 draft model trained for speculative decoding with Qwen3-0.6B as the target model.
All-time downloads
488
Public
Repo size
1.7 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors940 MB · 56%
At a glance
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Jul 10, 2026
- Updated
- Jul 10, 2026
- SHA
- a36bbdd8
Model card
From the Hugging Face model README
Qwen3-0.6B EAGLE-3 Draft Model
EAGLE-3 draft model trained for speculative decoding with Qwen3-0.6B as the target model.
Trained with a 60K Fortran-optimized vocabulary for 30 epochs on CPU (PyTorch), achieving:
- Train loss: 2.32
- Train accuracy: 79.7%
Files
| File | Format | Size | Description |
|---|---|---|---|
eagle3_draft_f16.gguf | GGUF F16 | 470 MB | Draft model in F16 precision |
eagle3_draft_q8.gguf | GGUF Q8_0 | 238 MB | Draft model quantized to Q8_0 |
hf_model/ | safetensors | 896 MB | Hugging Face format for fine-tuning |
Usage with llama.cpp
# Basic usage with speculative decoding
llama-cli \
-m Qwen3-0.6B-target.gguf \
-md eagle3_draft_q8.gguf \
--spec-type draft-eagle3 \
--spec-draft-n-max 5 \
-p "Hello, how are you?" \
-n 200
# Server mode (enables Ollama-like API)
llama-server \
-m Qwen3-0.6B-target.gguf \
-md eagle3_draft_q8.gguf \
--spec-type draft-eagle3 \
--spec-draft-n-max 5 \
--port 8080
Usage with Ollama
- Pull the target model:
ollama pull qwen3:0.6b
-
Download the draft model GGUF from this repo
-
Create a Modelfile:
FROM qwen3:0.6b
DRAFT_MODEL ./eagle3_draft_q8.gguf
PARAMETER speculative_type draft-eagle3
PARAMETER speculative_draft_max 5
- Create and run:
ollama create qwen3-eagle -f Modelfile
ollama run qwen3-eagle
Benchmark (CPU-only, 16GB RAM)
| Config | Speed (t/s) | vs Baseline |
|---|---|---|
| Baseline (no draft) | 33.2 | 1.00x |
| Ngram (len=3) | 34.6 | 1.04x |
| EAGLE-3 (len=3) | 15.6 | 0.47x |
| EAGLE-3 (len=5) | 11.8 | 0.36x |
Note: EAGLE-3 speedup requires GPU acceleration. On CPU-only, the dual-model overhead outweighs speculation benefits for small models (0.6B). Speedup expected with GPU offloading or larger target models.
Training Details
- Target model: Qwen3-0.6B (1024 dim, 3072 FFN, 28 layers)
- Draft architecture: EAGLE-3 with 1 Eagle3Block (1024 hidden, 8 heads, 4 KV heads)
- Draft vocabulary: 60K tokens (Fortran-optimized selection)
- Training data: 500 ShareGPT conversations
- Loss: Cross-entropy with cosine annealing + warmup
- Hardware: CPU-only (no CUDA), PyTorch 2.13.0
- Fortran integration: Acceptance sampling, logit correction, and rejection sampling kernels
Architecture
EAGLE-3 extracts hidden states from target model layers [2, 14, 25], projects them through a feature compressor, and feeds them into a lightweight transformer block for draft prediction.
- Library
- gguf
- License
- apache-2.0
- Created
- Jul 10, 2026
- Updated
- Jul 10, 2026