DeepSeek-V4-Flash-0731-REAP

DeepSeek V4 Flash REAP — UD-IQ1_S three-tier GGUF

This repository contains three exploratory, physically compacted expert-retention tiers derived from unsloth/DeepSeek-V4-Flash-0731-GGUF UD-IQ1_S at revision 1290dcca3f84612f646fb546fb9e8433c1b339b0.

Each variant keeps all 256 experts in protected hash-routing layers 0–2. Learned router layers 3–42 retain the experts selected by their fixed REAP prune plan:

Load a tier by selecting its first split, for example:

llama-server \
  -m DeepSeek-V4-Flash-0731-REAP-K224-00001-of-00003.gguf \
  -c 262144 -b 2048 -ub 512 -np 1 \
  --kv-offload -fa on -ctk q8_0 -ctv q8_0 -ngl 999

The pinned llama.cpp basis is a1f96d4fc2c9e4101a6666a9d87f547e7e880df6. These GGUFs encode deepseek4.expert_count as a per-layer array and therefore require the supplied patches/llama-cpp-deepseek4-per-layer-experts.patch. The supplied DeepSeek3 tokenizer long-run fix is also recommended. The REAP runtime patch is included for reproducing profiling and runtime-mask experiments.

Verification status

• All three compact writers completed with status=passed; their complete receipts and prune plans are published under receipts/ and prune-plans/.
• K192 and K224 were actually loaded with 262,144-token context, Q8 K/V cache, Flash Attention, and full GPU offload on an RTX PRO 6000 Blackwell 96GB.
• Observed process memory was 70,672 MiB for K192 and 79,006 MiB for K224.
• K192 served real requests after load; K224 reached health: ok.
• K160 has not yet received an equivalent real-load smoke in this publication.

This is an exploratory three-candidate publication requested before completion of the preregistered Pi/SWE-Lancer evaluation. It is not an RTX_EXPERIMENT_PASSED or formal HF publication acceptance result. Compact-vs- runtime-mask logit/output parity, final quality selection, A100 cold validation, and the formal agent evaluation remain pending. Do not treat smaller storage or VRAM as a demonstrated quality or speed improvement.

See SHA256SUMS for the public GGUF hashes. The original full model is not mirrored or replaced here.