Downloads · 30 days
130
12% of all-time downloads
ljupco/DeepSeek-V4-Flash-0731-REAP25-GGUF
DeepSeek-V4-Flash-0731-REAP25-GGUF is a text generation model from ljupco. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as mit.
A 25%-expert-pruned, 2-bit imatrix GGUF of deepseek-ai/DeepSeek-V4-Flash-0731, built for the antirez/ds4 (DwarfStar) engine and 96 GB Apple Silicon Macs.
Downloads · 30 days
130
12% of all-time downloads
All-time downloads
1.1K
Public
Repo size
68.6 GB
Likes
0
Public
Click a slice to open those files.
.gguf68.6 GB · 100%
From the Hugging Face model README
A 25%-expert-pruned, 2-bit imatrix GGUF of deepseek-ai/DeepSeek-V4-Flash-0731, built for the antirez/ds4 (DwarfStar) engine and 96 GB Apple Silicon Macs.
| File | DeepSeek-V4-Flash-0731-REAP25-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf |
| Size | 68,579,735,164 bytes (63.87 GiB) |
| Experts | 256 → 192 kept in layers 3–42 (REAP25, 25% pruned); layers 0–2 keep 256 |
| Quant | IQ2_XXS (gate/up) + Q2_K (down) routed experts; Q8_0 attention/shared/output; F16 HC/compressor/indexer |
| Engine | ds4 (--dspark optional) |
| Context | 1M (compressed KV: ~8.4 GiB at 1M) |
This is a GGUF-level prune of the stock 2-bit 0731 GGUF: the experts that the REAP25 saliency pruning dropped are removed from the file and the remaining bytes are recompacted. No re-quantization was done — every kept byte is byte-identical to the stock file, so quality is exactly the 2-bit quality of the source, minus the REAP pruning cost. The result is 68.58 GB and fits in 96 GB Macs that cannot hold the 80.8 GiB stock file.
REAP25 keeps 192 of the 256 experts in layers 3–42. In a GGUF, each expert's weights are stored as a separate contiguous, block-aligned chunk per layer (the expert axis is the slowest tensor dimension, and quant blocks never straddle expert boundaries). Pruning therefore means: for every layer, drop the exact 64 chunks that the REAP saliency run removed — not any 64. If the wrong chunks are dropped, the router addresses experts that are no longer there and the model produces garbage.
The REAP25 MLX weights are compacted to ids 0..191, so the original expert identity
is not stored in them. We did not find a published list of which experts were dropped
by the REAP saliency run, so we had to work out the mapping ourselves: each REAP25
expert's weights were matched against all 256 stock experts of the same layer on four
independent parts (gate/up/down expert weights + router rows). All four parts agreed
for 192/192 experts in 40/40 layers, with decisive margins — giving us the exact
deletion map used for this file.
gguf-tools/deepseek4-quantize).tools/reap25-prune/, docs/REAP25-REPRODUCE.md).reap.* metadata added
(reap.enabled, reap.layout=ds4-compact-v1, reap.layer.expert_count,
reap.layer.keep_count).DeepSeek-V4-Flash-DSpark-support.gguf) or
bleysg/DeepSeek-V4-Flash-DSpark-drafter-GGUF
(DSpark-drafter-Q2K-Q8-0731.gguf); load with --mtp <file> --dspark.Perplexity (131,040 wikitext-2-raw-v1 tokens, ds4, same protocol): stock 4.934 → REAP25 5.129, δ +0.195 ppl (+3.95%) — better than pipenetwork's published +4.5% at 4-bit (different protocol). A model property — engine updates do not change it.
Speed (pp 2048, tg 128, fixed 1M-ctx server; engine = the
reap-compact-support branch at 01bdc35, i.e. upstream DS4 rebased onto
2026-08-09 main 84cc882 with Ivan Fioravanti's exact pre-M5 Metal decode
fusions, 4e401a1 et al.):
| Context depth | Prefill (PP) t/s | Decode (TG) t/s |
|---|---|---|
| 1K | 183.6 | 31.7 |
| 8K | 192.0 | 35.2 |
| 32K | 111.0 | 15.6* |
| 128K | 93.2 | 24.5 |
*Samples are single-shot unless noted; expect ±a few %. The 32K decode sample was visibly stalled (mean 15.6 t/s vs peak 25.0); two same-session standard-build samples with the MTP head attached measured 29.96 / 30.46 t/s at 32K, so the true 32K decode is ≈28–30 t/s. 8K reproduced identically (35.19) across two separate server sessions.
The upstream long-context decode fix + pre-M5 fusions roughly doubled deep-context decode versus the pre-rebase engine (128K TG 11.23 → 24.47, +118%); 1K decode is unchanged (~32 t/s) and prefill is somewhat lower on this quantization path. Earlier Aug 3/4 numbers (TG 14.7/11.7/11.2/11.5/11.7 at 1K–128K) are superseded.
Full recipe (map recovery + GGUF surgery + validation): docs/REAP25-REPRODUCE.md
in the reap-compact-support
branch — DESIGN notes + RUNBOOK + scripts (bexact.py, prune_gguf.py,
extract_tables.py, keep_map_bexact.json).
--mtp <drafter> --dspark) is linked for
completeness but measures as a net slowdown on the M2 Max: the drafters
predict the pruned model well (~69% acceptance), yet proposal+verify overhead
exceeds the decode they save at every depth. Run without --dspark on this
class of hardware.This is small work sitting on top of a very large body of work by many people, and we want to thank them properly:
gguf-tools/deepseek4-quantize toolchain and imatrix, the REAP-compact runtime
support this file is built for, and the
DeepSeek-V4-Flash-DSpark-support.gguf
drafter.If you are one of the people above and we misrepresented your work, please open an issue or pull request on the reap-compact-support branch and we will fix it promptly.