Downloads · 30 days
5.9K
59% of all-time downloads
sleepyeldrazi/deepseek-v4-flash-reap-k128-Q2-GGUF
deepseek-v4-flash-reap-k128-Q2-GGUF is a text generation model from sleepyeldrazi. Use it when you need the model to write or continue text. The card lists the license as mit.
REAP-pruned DeepSeek V4 Flash at K128 (128 routed experts per MoE layer). Prunes 50% of routed experts via Cerebras REAP (Router-weighted Expert Activation Pruning), preserving all attention, embeddings, shared expert…
Downloads · 30 days
5.9K
59% of all-time downloads
All-time downloads
10K
Public
Repo size
50.4 GB
Likes
7
Public
Click a slice to open those files.
.gguf50.4 GB · 100%
From the Hugging Face model README
REAP-pruned DeepSeek V4 Flash at K128 (128 routed experts per MoE layer). Prunes 50% of routed experts via Cerebras REAP (Router-weighted Expert Activation Pruning), preserving all attention, embeddings, shared experts, router, and MTP components.
Drop-in compatible with the standard ds4-engine runtime — uniform IQ2_XXS/Q2_K expert quantization throughout all layers. No per-layer quant dispatch required.
| Base model | DeepSeek V4 Flash |
| Donor GGUF | antirez IQ2XXS-w2Q2K-AProjQ8 (80.8 GiB) |
| Pruning method | REAP (Cerebras Research) |
| Routed experts | 128 per layer (down from 256) |
| Kept slots | 5,888 / 11,008 |
| Hash-preserved | Layers 0-2 (256 experts each) |
| Pruned | Layers 3-42 (128 experts each) |
| Format | ds4-compact-v1 GGUF |
| File size | 46.98 GiB |
| Quantization | Uniform IQ2_XXS / Q2_K experts in all layers |
This is the recommended variant for most users. Uniform expert quantization means it works out of the box with any ds4-engine build. The mixed-precision variant (52 GiB) has Q4_K in layers 37-42 but requires runtime per-layer quant dispatch.
8,000 prompts · 5.0M tokens · 1.3B routed expert observations
| Domain | Share |
|---|---|
| Coding & development | 35-40% |
| Agentic tool-calling | 16% |
| Research & knowledge | 15-20% |
| Math & science | 10-15% |
| Design & planning | 5-10% |
| Trivia & general QA | 3-5% |
Calibration used the REAP activation_energy_sum2 score metric with 4,096 token context per prompt.
Top-to-bottom expert score gap in layer 3: 2,200x (strong pruning signal).
Requires eouya2/ds4-for-reaped (ds4 engine with compact GGUF support):
git clone https://github.com/eouya2/ds4-for-reaped
cd ds4-for-reaped
make cuda-spark -j$(nproc) # DGX Spark / CUDA
# or: make # Metal / macOS
./ds4 --cuda -m DeepSeek-V4-Flash-REAP-K128-uniform.gguf --ctx 131072
API server mode:
./ds4-server --cuda -m DeepSeek-V4-Flash-REAP-K128-uniform.gguf \
--host 0.0.0.0 --port 17777 --ctx 131072
activation_energy_sum2 (same calibration used for the mixed-precision variant — REAP scores are quantization-independent)ds4_prune_gguf.py from eouya2/reap-for-ds4. Layers 0-2 (hash-routed) preserved. Expert tensors copied byte-for-byte — no dequant/requant.ds4-compact-v1 GGUFNo fine-tuning. Purely structural expert removal. Weights are unmodified — a subset of the original experts.
| Uniform (this) | Mixed Precision | |
|---|---|---|
| File size | 46.98 GiB | 52.04 GiB |
| Expert quants | IQ2_XXS/Q2_K all layers | Q4_K in layers 37-42 |
| Compatibility | Drop-in | Needs quant-aware runtime |
| Quality | Baseline | Slightly higher in deep layers |