Downloads · 30 days
213
14% of all-time downloads
Lynxpda/DeepSeek-V4-Flash-0731-DSpark-Drafter-Q2_K_S-GGUF
DeepSeek-V4-Flash-0731-DSpark-Drafter-Q2_K_S-GGUF is a machine learning model from Lynxpda. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Optimized draft model for speculative decoding with DeepSeek-V4-Flash-0731. Quantized to Q2KS with minimal quality loss.
Downloads · 30 days
213
14% of all-time downloads
All-time downloads
1.6K
Public
Repo size
6.9 GB
Likes
1
Public
Click a slice to open those files.
.gguf6.9 GB · 100%
From the Hugging Face model README
Optimized draft model for speculative decoding with DeepSeek-V4-Flash-0731. Quantized to Q2_K_S with minimal quality loss.
Full speculative-decoding benchmark, target model
DeepSeek-V4-Flash-0731-UD-IQ3_XXS-q6kattn.gguf (96 GB, IQ3_XXS + Q6 attention),
7 task categories × 6 draft lengths × 5 runs each (210 requests per draft model).
Sampling: temp 0.9 / top_p 0.95 / min_p 0.01, ctx 65536, reasoning_effort=low.
Baseline (no draft model): 20.48 tok/s.
| Metric | Q8_0 draft | Q2_K_S draft (this) |
|---|---|---|
| File size | 10.15 GB | 6.45 GB (−36%) |
| Best overall decode | 28.43 tok/s | 28.58 tok/s |
| Best overall speedup vs baseline | 1.39× | 1.40× |
| Optimal n_max (overall) | 3 | 3 |
Q2_K_S matches the Q8_0 drafter within noise (±1–3% on every n_max and every
category, tie per the benchmark's own threshold) while being 36% smaller than
Q8_0 and 39% smaller than the BF16 drafter (10.54 GB).
Best result per category, averaged over Q8_0 + Q2_K_S drafts:
| Category | Best n_max | Decode tok/s | Speedup |
|---|---|---|---|
| code | 3 | 28.99 | 1.42× |
| prose | 2 | 23.38 | 1.14× |
| translate | 2 | 25.09 | 1.22× |
| json | 3 | 28.59 | 1.40× |
| math | 3 | 31.22 | 1.52× |
| repeat | 5 | 40.17 | 1.96× |
| dialog | 3 | 25.48 | 1.24× |
| overall | 3 | 28.50 | 1.39× |
Note on the averages. The 28.5 tok/s overall figure is pulled up by the mix: the suite is heavy on tasks that speculative decoding likes (repeat, math, code). Prompts also used a relatively short context (up to ~32k tokens of the 64k window). On a more typical mixed workload — especially with a fuller context — expect closer to 22–28 tok/s.
| n_max | Decode tok/s | Speedup | Accept rate |
|---|---|---|---|
| 2 | 27.12 | 1.32× | 0.693 |
| 3 | 28.50 | 1.39× | 0.602 |
| 4 | 27.73 | 1.35× | 0.522 |
| 5 | 26.43 | 1.29× | 0.455 |
| 6 | 26.40 | 1.29× | 0.456 |
| 7 | 26.52 | 1.29× | 0.451 |
n_max=3 is the sweet spot: longer drafts (n_max≥5) buy almost nothing and even start to hurt on prose/translate/dialog (low acceptance → wasted draft tokens).
Run the server with --spec-draft-n-max 3. If the workload is
code/json/math-heavy, n_max=3 is optimal; for repeat-like (high-acceptance)
loads, n_max=5–6 adds ~1.5× more. The Q2_K_S drafter is the drop-in choice:
same speed as Q8_0 at 36% smaller footprint. Treat the 28.5 tok/s headline as
an optimistic mix, not a day-to-day guarantee; 22–28 tok/s is the realistic band.
Full chart set in bench_reports/20260817_225557/charts/:






Also in the same directory: per-draft decode charts (decode_tps_by_category_q8.png,
_q2.png, overlay variant), per-draft heatmaps (speedup_heatmap_q8.png, _q2.png),
decode_tps_vs_nmax_avg.png, speedup_grouped_bars.png, speedup_vs_accept_rate.png.
-md / --spec-type on the same target modelllama-server from strix-halo-llamacpp, -fa 1, -ngld 999, -np 1, draft-dspark, ngram-mod offQuantized using upstream llama.cpp llama-quantize with --allow-requantize and custom tensor-type overrides.
Expert weights (92% of model):
Attention & shared experts:
Protected: