Downloads · 30 days
602
46% of all-time downloads
pokey2025/dspark-DeepSeek-V4-Pro-0813.gguf
dspark-DeepSeek-V4-Pro-0813.gguf is a text generation model from pokey2025. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as mit.
GGUF conversion of the dspark speculative-decoding drafter from deepseek-ai/DeepSeek-V4-Pro-0813 for use as a draft model with llama.cpp or ikllama.cpp's --spec-type dspark.
Downloads · 30 days
602
46% of all-time downloads
All-time downloads
1.3K
Public
Repo size
30.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf30.8 GB · 100%
From the Hugging Face model README
GGUF conversion of the dspark speculative-decoding drafter from
deepseek-ai/DeepSeek-V4-Pro-0813
for use as a draft model with llama.cpp
or ik_llama.cpp's --spec-type dspark.
dspark-DeepSeek-V4-Pro-0813-IQ3_XXS-imx.gguf — IQ3_XXS draft model (29 GB)Attach to a DeepSeek-V4-Pro-0813 verifier with --spec-type dspark:n_max=5
(the dspark config advertises dspark_block_size: 5):
llama-server \
--model /path/to/DeepSeek-V4-Pro-0813-IQ3_XXS-00001-of-00015.gguf \
--model-draft /path/to/dspark-DeepSeek-V4-Pro-0813-IQ3_XXS-imx.gguf \
--spec-type dspark:n_max=5
The drafter is a standard DFLASH-arch GGUF loaded via --spec-type dspark, so
it should work with both llama.cpp and
ik_llama.cpp (a llama.cpp fork); the
dspark loader expects this drafter's specific tensor names (fc /
enc.output_norm / markov_w1 / markov_w2 / conf_proj).
Tested only on ik_llama.cpp, chosen specifically to fit the full 1M context
via --swa-compress (which upstream llama.cpp does not have). It has not been
tested on other inference engines.
blk.{0,1,2}.ffn_{gate,down,up}_exps.weight,
~96% of the drafter) are IQ3_XXS (3.06 bpw); the attention / markov / fc /
shexp tensors stay at their original Q8_0 / BF16 / F32. This was a deliberate
pre-emptive size cut so the 29 GB drafter + ~53 GB IQ3_XXS main model fits
within 96 GB VRAM with headroom (the full 41 GB MXFP4 drafter would land at
~94 GB, right at the ceiling).-imx suffix means "imatrix-quantized". IQ3_XXS requires an importance
matrix; the imatrix used is the 6block main-model imatrix
(6block/DeepSeek-V4-Pro-0813-GGUF → imatrix.gguf), whose blk.0/1/2
expert entries are shape-compatible with the drafter's 3 MTP layers and have
clean statistics. The drafter's own target layers (main model blk.58-60)
had pathological imatrix entries (zero-count NaNs, extreme outliers) and were
deliberately not used.How this drafter GGUF was produced (for reproducibility / re-quantization):
Download the 3 dspark safetensors shards (all mtp.*-prefixed tensors) +
config/tokenizer from deepseek-ai/DeepSeek-V4-Pro-0813:
model-00064-of-00066.safetensors, model-00065-of-00066.safetensors,
model-00066-of-00066.safetensors
Write a trimmed model.safetensors.index.json containing only the 7009
mtp.* tensors so the converter opens only the 3 shards.
Convert to BF16 GGUF with upstream llama.cpp's convert_hf_to_gguf.py
(--dspark is only in upstream ggml-org/llama.cpp, not ik_llama.cpp):
python convert_hf_to_gguf.py <dir> --outtype bf16 \
--outfile dspark-DeepSeek-V4-Pro-0813-BF16.gguf \
--dspark --target-model-dir <dir> --model-name DeepSeek-V4-Pro-0813
→ 81 tensors, 42.1 GB BF16.
Quantize with ik_llama.cpp llama-quantize, requantizing only the 9 expert
tensors to IQ3_XXS (imatrix required) and pinning every other weight to its
current type via a --tensor-type-file:
# expert-tt.txt: 9 experts -> IQ3_XXS; all other weights -> current type
llama-quantize --allow-requantize \
--imatrix imatrix-6block-main.gguf \
--tensor-type-file expert-tt.txt \
dspark-DeepSeek-V4-Pro-0813-BF16.gguf \
dspark-DeepSeek-V4-Pro-0813-IQ3_XXS-imx.gguf IQ3_XXS 64
→ 29 GB (9 experts IQ3_XXS @ 3.24 GB each; ~1.6 GB rest preserved).
Tested on:
Adjust the ${modelsPath} prefix to wherever you download the weights — place
both the base-model shards and the drafter under
${modelsPath}/DeepSeek-V4-Pro-0813/:
/etc/ik-llama-cpp/llama-server
--model ${modelsPath}/DeepSeek-V4-Pro-0813/DeepSeek-V4-Pro-0813-IQ3_XXS-00001-of-00015.gguf
--model-draft ${modelsPath}/DeepSeek-V4-Pro-0813/dspark-DeepSeek-V4-Pro-0813-IQ3_XXS-imx.gguf
--spec-type dspark:n_max=5
--jinja
-amb 512
-ctk f16
-ctv f16
-ngl 99
--cpu-moe
--swa-compress
--ctx-size 1000000
--batch-size 8192
--ubatch-size 8192
--threads 64
--ctx-checkpoints 12
Measured with ik_llama.cpp (IQ3_XXS main model, 1M ctx):