Downloads · 30 days
146
42% of all-time downloads
Llama-3.2-1B-Instruct-DSpark
rasyosef/Llama-3.2-1B-Instruct-DSpark
Llama-3.2-1B-Instruct-DSpark is a machine learning model from rasyosef. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama3.2.
A DSpark draft model for speculative decoding with unsloth/Llama-3.2-1B-Instruct as the verifier, trained with speculators. The drafter proposes 8 tokens at a time and the verifier checks them in one forward pass, so…
All-time downloads
345
Public
Parameters
302M
2.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors604 MB · 100%
Parameter types
How the weights are stored.
BF16302M · 100%
Base models
Model card
From the Hugging Face model README
Llama-3.2-1B-Instruct-DSpark
A DSpark draft model for speculative decoding with unsloth/Llama-3.2-1B-Instruct as the verifier, trained with speculators. The drafter proposes 8 tokens at a time and the verifier checks them in one forward pass, so output is identical to running the verifier alone — a lossless speedup. Mean acceptance length is 3.15 tokens committed per verification step, up to 4.57 on HumanEval.
Training code: rasyosef/train-dspark-draft-models.
Trained on 100,000 samples.
Usage
vLLM
vLLM loads the verifier automatically from the config — don't pass it separately.
vllm serve rasyosef/Llama-3.2-1B-Instruct-DSpark --port 8000 --gpu-memory-utilization 0.8
Then query the OpenAI-compatible endpoint at http://localhost:8000/v1.
llama.cpp
llama-server -hf unsloth/Llama-3.2-1B-Instruct-GGUF:Q8_0 \
-hfd rasyosef/Llama-3.2-1B-Instruct-DSpark-GGUF:Q8_0 \
--spec-type draft-dspark --spec-draft-n-max 8 --spec-draft-n-min 0 \
-fa on -ngl 99
The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact: the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted.
Details
3 Qwen3 layers (hidden size 2048, intermediate size 8192, 32 attention heads over 8 KV heads, sliding-window attention with a 2048-token window), ~0.3B params, bfloat16. Block size 8, draft vocabulary reduced to 32,000, aux hidden-state layers 2/8/14, confidence head with Markov (rank 256).
Trained for 3 epochs at lr 3e-4 (AdamW, cosine schedule with 4% warmup) on 100,000 Open PerfectBlend prompts regenerated by the verifier itself, split 96/4 into train and validation, with a {"ce": 0.1, "tv": 0.9} loss. Prompts prepared at 1024 tokens; training sequence length 8192, up to 1024 anchors per sample. Verifier hidden states were pulled on demand from a running vLLM server during training and deleted after use rather than staged to disk up front. speculators 0.8.0.dev207, vLLM 0.28.0, transformers 5.15.1, torch 2.13.0.
Evaluation
evaluate.py throughput across the nine RedHatAI/speculator_benchmarks subsets. acceptance_length is mean tokens committed per verification step, including the bonus token — floor 1.0, ceiling 9.0 at block size 8.
| subset | acceptance_length | pos_0 | pos_1 | pos_2 | pos_3 | pos_4 | pos_5 | pos_6 | pos_7 |
|---|---|---|---|---|---|---|---|---|---|
| HumanEval | 4.573 | 83.7% | 68.5% | 56.2% | 46.0% | 36.8% | 28.3% | 21.7% | 16.1% |
| math_reasoning | 4.436 | 83.6% | 68.2% | 55.0% | 43.8% | 34.5% | 25.8% | 19.2% | 13.4% |
| tool_call | 3.462 | 73.3% | 55.5% | 41.3% | 30.1% | 20.2% | 13.3% | 8.1% | 4.3% |
| question | 2.646 | 63.0% | 38.9% | 24.3% | 15.8% | 9.7% | 6.4% | 4.0% | 2.5% |
| writing | 2.636 | 61.7% | 38.8% | 24.4% | 15.8% | 9.9% | 6.2% | 4.1% | 2.6% |
| rag | 2.523 | 64.9% | 39.4% | 23.9% | 13.3% | 6.4% | 2.8% | 1.1% | 0.5% |
| qa | 2.258 | 55.4% | 32.1% | 18.2% | 9.6% | 5.2% | 2.8% | 1.6% | 0.9% |
| summarization | 2.182 | 58.5% | 31.0% | 16.3% | 7.6% | 3.2% | 1.1% | 0.4% | 0.1% |
| translation | 2.021 | 54.6% | 28.6% | 12.6% | 4.3% | 1.3% | 0.5% | 0.2% | 0.0% |
Weighted across all subsets: 3.148 over 89,102 verification steps.
Acceptance is still highest where the verifier's next token is most predictable — code, math, structured tool calls. HumanEval and math_reasoning are far ahead of everything else and hold their lead deep into the block: HumanEval's pos_4 (36.8%) is above summarization's pos_1 (31.0%), and both still accept better than one token in six at pos_7. The prose-like subsets cluster tightly at 2.0–2.6 and fall off sharply after pos_3, where the longer block buys little — translation is under 2% accepted from pos_4 onward.
Limitations
Works only with Llama-3.2-1B-Instruct and is not usable as a standalone model. Acceptance falls off steeply past the first few positions on prose-like traffic (summarization, qa, translation), so a block size of 8 is mostly wasted there — the gains concentrate in code, math, and tool calls. Real-world speedup depends on your traffic mix, and because verification is lossless, the verifier's own behavior and biases carry through unchanged.
License
Llama 3.2 Community License, inherited from the verifier. The speculators training code is Apache-2.0.
- Library
- transformers
- License
- llama3.2
- Created
- Aug 30, 2026
- Updated
- Oct 5, 2026