Downloads · 30 days
21
30% of all-time downloads
qwen3-0.6b-speculator
selimaktas/qwen3-0.6b-speculator
qwen3-0.6b-speculator is a machine learning model from selimaktas. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is a Qwen3-0.6B model fine-tuned to serve as a speculative decoder for Qwen3-14B, enabling faster inference through speculative decoding.
All-time downloads
71
Public
Parameters
596M
1.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.2 GB · 99%
At a glance
- Model type
- qwen3
- Access
- Public
- Created
- Sep 19, 2025
- Updated
- Sep 19, 2025
- SHA
- e822ce71
Model card
From the Hugging Face model README
Qwen3-0.6B Speculative Decoder for Qwen3-14B
This is a Qwen3-0.6B model fine-tuned to serve as a speculative decoder for Qwen3-14B, enabling faster inference through speculative decoding.
Model Details
- Base Model: Qwen/Qwen3-0.6B
- Target Model: Qwen/Qwen3-14B
- Training: Knowledge distillation with ArcticTraining
- Use Case: Speculative decoding for 2-4x inference speedup
Usage
With vLLM + Arctic Inference
from vllm import LLM
llm = LLM(
model="Qwen/Qwen3-14B",
speculative_model="your-username/qwen3-14b-speculator",
num_speculative_tokens=3,
trust_remote_code=True
)
outputs = llm.generate("Hello, how are you?", max_tokens=100)
print(outputs[0].outputs[0].text)
- Type
- qwen3
- Created
- Sep 19, 2025
- Updated
- Sep 19, 2025