Downloads · 30 days
32
1% of all-time downloads
Ralii/DeepSeek-V4-Flash-0731-DSpark-Selective-Q4_K_M-GGUF
DeepSeek-V4-Flash-0731-DSpark-Selective-Q4_K_M-GGUF is a text generation model from Ralii. Use it when you need the model to write or continue text. The card lists the license as mit.
A selectively quantized DSpark speculative-decoding sidecar for DeepSeek-V4-Flash-0731, prepared and tested with llama.cpp Vulkan on AMD Strix Halo.
Downloads · 30 days
32
1% of all-time downloads
All-time downloads
2.7K
Public
Repo size
10.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf10.6 GB · 100%
From the Hugging Face model README
A selectively quantized DSpark speculative-decoding sidecar for DeepSeek-V4-Flash-0731, prepared and tested with llama.cpp Vulkan on AMD Strix Halo.
This is not a standalone language model, a newly trained drafter, or a new
speculative-decoding architecture. It is a derived quantization of
dspark-DeepSeek-V4-Flash-0731-MXFP4.gguf.
It must be used with the matching DeepSeek-V4-Flash-0731 target.
| Filename | Size | SHA-256 |
|---|---|---|
dspark-DeepSeek-V4-Flash-0731-MXFP4-Q4_K_M-dense.gguf | 10,571,389,760 bytes | 284adb03bc0f52576ac612a0e676f33cf3c218e26c8fcb18c877e571287c1be0 |
Tensor types:
The source sidecar is 10,896,059,200 bytes with SHA-256
2dbdc67c9cf69d88783c32cc17dcf59a52e2b86ee46c5df6ad737fb9fe26a9f1.
The nine routed-expert gate/up/down tensors were kept in their original MXFP4 format. Eligible Q8_0, BF16, and F32 dense tensors were quantized using llama.cpp's Q4_K_M mixture.
A whole-file Q4_K_M conversion was rejected because it requantized the already compact MXFP4 experts and produced a larger sidecar. This selective conversion reduced the file by approximately 3% and was faster than the original sidecar on the tested Strix Halo Vulkan configuration.
No weights were trained or fine-tuned.
Tested with:
UD-IQ3_XXSThe sidecar should be compatible with other quantizations of the exact DeepSeek-V4-Flash-0731 target, but acceptance and performance can vary with the target quantization, backend, sampler, workload, and context depth.
Use a recent llama.cpp build supporting draft-dspark and
--spec-draft-p-min. The exact benchmark build used the Strix Halo Vulkan
lineage from
Nathanw1014/llama.cpp, based on
commit 7a57bedaaed78d1fa4047eb6e7011b0a80f302d5, plus a local DeepSeek-V4
wo_a graph patch. That patch improves the tested target runtime; it does not
alter this GGUF.
Download with the Hugging Face CLI:
hf download \
Ralii/DeepSeek-V4-Flash-0731-DSpark-Selective-Q4_K_M-GGUF \
dspark-DeepSeek-V4-Flash-0731-MXFP4-Q4_K_M-dense.gguf \
--local-dir .
Verify the artifact:
echo '284adb03bc0f52576ac612a0e676f33cf3c218e26c8fcb18c877e571287c1be0 dspark-DeepSeek-V4-Flash-0731-MXFP4-Q4_K_M-dense.gguf' \
| sha256sum --check
llama-server \
-m /path/to/DeepSeek-V4-Flash-0731-target.gguf \
-ngl 99 \
-fa on \
-ctk f16 -ctv f16 \
-c 131072 \
-b 2048 -ub 1024 \
--parallel 1 \
--spec-type draft-dspark \
--spec-draft-model ./dspark-DeepSeek-V4-Flash-0731-MXFP4-Q4_K_M-dense.gguf \
--spec-draft-n-max 5 \
--spec-draft-p-min 0.45
Example sampled reasoning settings used in production:
--reasoning on --temp 1.0 --top-p 1.0 --min-p 0.05
These sampler settings are not universally optimal. Keep deterministic and sampled reasoning benchmarks separate.
Using llama.cpp's llama-quantize:
llama-quantize \
--allow-requantize \
--tensor-type 'blk\.[0-2]\.ffn_(gate|up|down)_exps\.weight=mxfp4' \
dspark-DeepSeek-V4-Flash-0731-MXFP4.gguf \
dspark-DeepSeek-V4-Flash-0731-MXFP4-Q4_K_M-dense.gguf \
Q4_K_M 16
The output should match the published size and SHA-256 above when created with the same source artifact and compatible quantizer implementation.
Four token-identical real coding prompts at approximately 16K context, temperature zero, and a maximum of 384 generated tokens per task:
| Draft | Weighted generation speed | Acceptance |
|---|---|---|
| Original MXFP4/Q8 sidecar | 18.95 t/s | 52.8% |
| Selective-Q4 sidecar | 20.28 t/s | 55.9% |
This was approximately 7% faster than the original DSpark sidecar in that comparison. It is a hardware- and workload-specific result, not a universal performance claim.
A larger deterministic campaign used 62 runnable Clojure coding tasks with token-identical prompts, reasoning disabled, and a 384-token output cap. DSpark used block 5 and confidence floor 0.45.
| Active context | Raw target | Selective-Q4 DSpark | Change |
|---|---|---|---|
| 16K | 17.78 t/s | 23.65 t/s | +33.0% |
| 64K | 16.22 t/s | 17.24 t/s | +6.3% |
| 72K | 15.90 t/s | 16.64 t/s | +4.6% |
| 80K | 15.77 t/s | 15.96 t/s | +1.3% |
| 84K | 15.59 t/s | 15.63 t/s | +0.3% |
| 90K | 15.67 t/s | 15.36 t/s | -1.9% |
On this system, 72K was the last comfortably positive deterministic point and 80–84K was approximately break-even. Sampled reasoning was more trajectory-dependent. Benchmark against raw decoding before enabling the sidecar globally, especially at long context.
Higher acceptance does not necessarily mean higher throughput: shorter blocks increased acceptance at 90K but were substantially slower.
The target model remains authoritative during speculative decoding: rejected draft tokens are not committed. Nevertheless, target quantization, sampling, EOS behavior, and workload can change trajectories and acceptance economics. Validate the complete application rather than relying on an acceptance ratio.
The development campaign used runnable hidden tests, nREPL validation, and external coding quality gates. No claim is made that these measurements cover every language or workload.
A separate 5.99 GB DS4-style IQ2_XXS/Q2_K experiment was not published as the recommended artifact: it was 0.89% slower in repeated deterministic tests and 5.66% slower with the production sampler on the existing Vulkan kernels.
Source patches, commands, tested settings, results, and limitations are documented at:
https://github.com/Ralii/strix-halo-llamacpp/tree/strix-halo-deepseek-v4-adaptive
This repository does not imply endorsement by those projects.
The original DeepSeek-V4-Flash-0731 repository and weights are licensed under
the MIT License. This derived GGUF is redistributed under the same license.
The original copyright and license notice are included in LICENSE.
@misc{deepseekai2026deepseekv4,
title = {DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
author = {DeepSeek-AI},
year = {2026}
}