Downloads · 30 days
40
9% of all-time downloads
shi0203/DeepSeek-V4-Flash-0731-K160-REAP-GGUF
DeepSeek-V4-Flash-0731-K160-REAP-GGUF is a text generation model from shi0203. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as mit.
Experimental model This is an unofficial REAP K160 compact variant of DeepSeek V4 Flash 0731. Quality evaluation is still in progress, and some capabilities are significantly degraded compared with the full model.
Downloads · 30 days
40
9% of all-time downloads
All-time downloads
466
Public
Repo size
58.8 GB
Likes
1
Public
Click a slice to open those files.
.gguf58.8 GB · 100%
From the Hugging Face model README
Experimental model
This is an unofficial REAP K160 compact variant of DeepSeek V4 Flash 0731. Quality evaluation is still in progress, and some capabilities are significantly degraded compared with the full model.
This repository contains a REAP K160 / 160-expert compact variant of DeepSeek V4 Flash 0731, created by pruning routed experts directly from an already-working quantized GGUF.
The current Q2 variant starts from the asymmetric / mixed Q2 quantization published by antirez/deepseek-v4-gguf, and removes the routed experts excluded by the K160 REAP expert selection from 0xSero/DeepSeek-V4-Flash-0731-REAP.
The retained expert weights are not dequantized and requantized.
In simplified form:
DeepSeek-V4-Flash-0731
│
├── antirez Q2 GGUF
│ │
│ └── keep already-quantized weights
│
└── 0xSero REAP K160 expert selection
│
└── remove pruned experts
│
▼
This K160 compact Q2 GGUF
The goal is to reduce memory usage while preserving as much of the behavior of the known-working Q2 quantization as possible.
There are already quantized REAP K160 variants available.
However, this model takes a different conversion path.
Instead of starting from an already-pruned K160 checkpoint and quantizing its remaining weights again, this variant starts from an existing working Q2 GGUF and only removes the experts excluded by the K160 pruning map.
This avoids an additional:
quantized / mixed-precision weights
↓
dequantization
↓
requantization to Q2
step for the retained expert weights.
The practical motivation is simple: I want a K160 model small enough to run comfortably on memory-constrained local hardware while remaining useful for real agentic workloads.
The model is still under evaluation.
More results will be added as testing progresses.
Japanese capability is effectively unusable in my testing after REAP K160 pruning.
Normal Japanese conversation cannot be reliably sustained.
This is not a small quality regression; for practical purposes, do not use this model for Japanese-language workloads.
English agentic use is the primary target of my current testing.
The current Q2 variant is derived from the imatrix quantization in:
The original quantization uses an asymmetric mixed-precision layout approximately structured as follows:
| Tensor group | Quantization |
|---|---|
| Routed expert gate/up | IQ2_XXS |
| Routed expert down | Q2_K |
| Attention projections | Q8_0 |
| Shared experts | Q8_0 |
| Output | Q8_0 |
| Other tensors | Mixed higher precision |
This repository preserves the already-quantized tensors of the retained experts rather than requantizing them.
DeepSeek-V4-Flash-0731-K160-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf
REAP pruning metadata / plan:
DeepSeek-V4-Flash-0731-K160-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-imatrix.gguf.reap_plan.json
Additional quantization variants may be added to this repository later.
In particular, a ROCmFP2 variant is under consideration.
This model uses a compact REAP K160 layout.
The routed expert tensors physically contain fewer expert slots than the original DeepSeek V4 Flash model, so a runtime with explicit compact REAP support is required.
| Backend | Runtime |
|---|---|
| Metal / CUDA / CPU | eouya2/ds4-for-reaped |
| ROCm | shin902/ds4-rocm |
Use:
https://github.com/eouya2/ds4-for-reaped
This DS4 fork adds support for compact REAP GGUF layouts, including routed-expert tensors whose physical expert dimension has been reduced.
Use:
https://github.com/shin902/ds4-rocm
This is my DS4 fork with ROCm support for the compact REAP model layout.
It is the runtime used for my current testing on AMD Strix Halo.
The original inference engine is:
However, upstream DS4 currently does not directly load this compact K160 GGUF.
The original model stores 256 routed experts, while this model physically stores only 160. Attempting to load this compact model with an incompatible upstream build results in a tensor shape validation error such as:
ds4: tensor blk.0.ffn_gate_inp.weight has dim[1]=160, expected 256
Use one of the REAP-compatible forks listed above instead.
My primary test system is:
AMD Ryzen AI MAX+ 395 (Strix Halo)
96 GB unified memory
ROCm
Linux
The broader goal is to make DeepSeek V4 Flash practically usable on hardware where the full-size model leaves too little memory headroom for context, KV cache, agent workloads, and other system processes.
I plan to compare:
Full asymmetric Q2
vs
This REAP K160 Q2
vs
Other K160 quantizations
Across several dimensions:
The purpose is not only to ask whether the model "runs", but whether the resulting compressed model remains practically usable.
deepseek-ai/DeepSeek-V4-Flash-0731
0xSero/DeepSeek-V4-Flash-0731-REAP
The K160 expert selection is used to determine which routed experts are retained.
The actual retained expert weights in this Q2 release come from the already-quantized antirez Q2 GGUF rather than from quantizing the 0xSero K160 checkpoint.
MIT.
DeepSeek V4 Flash 0731 is released under the MIT License.
This repository is an unofficial derivative built using resources from the projects referenced above. See each upstream repository for its original license notices and attribution information.
This is an unofficial experimental derivative.
It is not an official release from DeepSeek or the upstream projects referenced above.
Compatibility information, benchmark results, model quality, and recommended runtimes may change as the model and inference software continue to evolve.