Downloads · 30 days
57
8% of all-time downloads
heath0xFF/DeepSeek-V4-Flash-0731-REAP-K216-GGUF
DeepSeek-V4-Flash-0731-REAP-K216-GGUF is a machine learning model from heath0xFF. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as mit.
An experimental, single-DGX-Spark build of DeepSeek V4 Flash 0731 that retains 216 of 256 routed experts per MoE layer.
Downloads · 30 days
57
8% of all-time downloads
All-time downloads
693
Public
Repo size
89.1 GB
Likes
3
Public
Click a slice to open those files.
.gguf89.1 GB · 100%
From the Hugging Face model README
An experimental, single-DGX-Spark build of DeepSeek V4 Flash 0731 that retains 216 of 256 routed experts per MoE layer.
This repository uses a different construction order from earlier K160 conversions:
This preserves the stock quantization exactly for all retained tensors and avoids quantizing an already transformed/repacked REAP checkpoint.
| Variant | Size | Status |
|---|---|---|
DeepSeek-V4-Flash-0731-REAP-K216-UD-IQ3_XXS-00001-of-00003.gguf | 82.94 GiB total | Validated on NVIDIA GB10 |
The first shard is the entry point; llama.cpp discovers the remaining shards automatically.
UD-IQ3_XXS22f28d32b9b29b4352eaa380ff8c2c170b2847abThe included structured report and plan record the selected experts. The included validator compared all 1,328 output tensors and all 89,060,075,612 tensor-data bytes against the stock quant and K216 routing source:
tid2eid routing tables matched 0xSero's carried tensors exactly.Tested on an NVIDIA DGX Spark / GB10 with llama.cpp b10223 (11924d4c1), CUDA architecture 121a.
MUL_MAT_ID correctness tests passed for the IQ2_XS gate/up projections and IQ3_XXS down projections with 216 experts and top-6 routing.Configured-context smoke tests used one slot, Q8_0 K/V cache, full CUDA offload, -b 1024 -ub 512, and the same short non-thinking API prompt:
| Configured context | Prompt | Generation | CUDA allocation | Result |
|---|---|---|---|---|
| 32,768 | 46.8 tok/s | 16.8 tok/s | 85,192 MiB | Pass |
| 65,536 | 48.1 tok/s | 16.6 tok/s | 85,330 MiB | Pass |
| 131,072 | 44.1 tok/s | 16.3 tok/s | 85,640 MiB | Pass |
| 262,144 | — | — | 86,330 MiB | Initializes; first prompt crashes in the current llama.cpp CUDA path |
Recommended GB10 context: 131,072 tokens. This is the largest configuration currently validated to initialize and generate successfully. These are short-prompt allocation/generation smoke tests, not tests with the entire context window occupied; sustained long-context quality and memory behavior still need evaluation.
Use a recent llama.cpp build with deepseek4 support. On GB10, mmap is recommended; --no-mmap needlessly duplicates memory pressure for this model.
llama-cli \
-m DeepSeek-V4-Flash-0731-REAP-K216-UD-IQ3_XXS-00001-of-00003.gguf \
-ngl 999 \
--flash-attn on \
-c 131072 \
-b 1024 \
-ub 512 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--temp 1.0 \
--top-p 1.0 \
--top-k 0 \
--min-p 0 \
--repeat-penalty 1.0 \
-n 256 \
-p "Write one sentence about the moon."
DeepSeek V4 Flash defaults to a thinking-style template. For direct-answer diagnostics, disable reasoning when your llama.cpp client or API exposes that option. Do not treat a repetition penalty as a fix for a conversion or template problem.
tools/reap_k216_gguf.py performs the byte-preserving rewrite.tools/validate_reap_k216_gguf.py performs exhaustive tensor-data validation.metadata/REAP_K216_PLAN.json records the rewrite plan.metadata/REAP_K216_STRUCTURED_REPORT.json records 0xSero's structured K216 selection.This is frontier/experimental work. Please include your llama.cpp commit, backend, hardware, context, exact command, and minimal prompt when reporting results.
MIT, inherited from the source model.
Thanks to DeepSeek, Unsloth, 0xSero, the REAP authors, and llama.cpp contributors.