Downloads · 30 days
132
5% of all-time downloads
jmilnz/DeepSeek-V4-Flash-0731-antirez-ds4-GGUF
DeepSeek-V4-Flash-0731-antirez-ds4-GGUF is a text generation model from jmilnz. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as mit.
Mixed-precision GGUFs of DeepSeek-V4-Flash-0731, built for antirez/ds4 (DwarfStar) on 128 GB Apple Silicon. Reproduces the canonical antirez "layers 37-42 Q4 / everything else Q2" recipe, applied to the 0731 weights.
Downloads · 30 days
132
5% of all-time downloads
All-time downloads
2.5K
Public
Repo size
195 GB
Likes
5
Public
Click a slice to open those files.
.gguf97.4 GB · 100%
From the Hugging Face model README
Mixed-precision GGUFs of DeepSeek-V4-Flash-0731, built for antirez/ds4 (DwarfStar) on 128 GB Apple Silicon. Reproduces the canonical antirez "layers 37-42 Q4 / everything else Q2" recipe, applied to the 0731 weights.
DeepSeek-V4-Flash-0731-Layers37-42Q4K-mixed-realimatrix-v2.gguf — 91 GiB
(97,426,596,736 bytes), quantized with a real activation imatrix.
An earlier build of this same recipe used the quantizer's synthetic weight-energy fallback instead. It has been removed in favour of this one, which measures better on every metric tested (see Quality below). The two had an identical tensor-type layout and identical runtime cost — they differed only in which quantization codes were chosen for the routed experts.
deepseek-ai/DeepSeek-V4-Flash-0731 ships as ~155 GB of fp8 safetensors — too
large for a 128 GB Mac alongside KV cache and the OS. These files compress the
0731 snapshot to ~91 GB using antirez's mixed-precision strategy, so they drop
straight into an existing ds4 setup.
| Tensors | Quant | Rationale |
|---|---|---|
| Routed MoE experts, layers 37–42 (the "hot" layers) | Q4_K | Higher fidelity where it matters most |
| Routed MoE experts, all other layers | IQ2_XXS (~2-bit) | The cold bulk, aggressively compressed |
| Down projections | Q2_K | 2-bit |
| Attention projections | Q8_0 | Small, fires every token — keep sharp |
| Shared experts | Q8_0 | Always-on dense path |
| Output head | Q8_0 | Critical, small |
Verified layout: 18 × Q4_K + 37 × Q2_K + 74 × IQ2_XXS = 129 routed-expert tensors across 43 layers.
v2 is built in a single deepseek4-quantize pass, using --tensor-type
overrides to promote layers 37-42 rather than quantizing a separate ~153 GiB Q4
donor and splicing it in:
args=()
for L in 37 38 39 40 41 42; do
for T in gate down up; do
args+=(--tensor-type "blk.$L.ffn_${T}_exps.weight=q4_k")
done
done
gguf-tools/deepseek4-quantize \
--hf /path/to/DeepSeek-V4-Flash-0731 \
--template DeepSeek-V4-Flash-0731-Q2-imatrix.gguf \
--imatrix DeepSeek-V4-Flash-0731-routed-moe-real.dat \
--out DeepSeek-V4-Flash-0731-Layers37-42Q4K-mixed-realimatrix-v2.gguf \
"${args[@]}" --threads 16
This produces a byte-identical layout to the splice route (same 1328 tensors, same 18 type changes) while skipping an entire quantize pass.
v2 uses a real 0731-native routed-MoE imatrix collected with
ds4 --imatrix-out (129 entries = 43 layers × {gate, down, up}, 2752 chunks —
comparable calibration depth to antirez's own reference imatrix). v1 used the
quantizer's synthetic weight-energy fallback.
Do not pass --imatrix-strict with a routed-MoE imatrix. The first v2 attempt
did, and produced a silently truncated 73 GiB file. The flag means "fail if a
quantized tensor has no matching imatrix vector" — but a routed-MoE imatrix by
construction only covers routed experts. It therefore exit(1)s on
token_embd.weight, the first non-expert tensor in write order, at byte offset
77,922,699,264 (72.6 GiB), killing the process mid-write. This is deterministic:
strict mode can never succeed with a routed-MoE imatrix. Plain --imatrix is
correct usage — the fallback for non-expert tensors is the intended path, not
silent data loss.
Two related gotchas:
chat-v2 GGUF
instead of the 0731 Q2 shifts the planned size by ~165 MB.ls -lh prints GiB. A correct file is
97,426,596,736 B, which ls -lh shows as 91G. A real-imatrix build is also
exactly 288 bytes larger than a non-imatrix one — those are the four
quantize.imatrix.* provenance keys, not corruption.Scored with gguf-tools/quality-testing/score_official against the 100 tracked
DeepSeek API continuations in data/flash, compared via compare_scores.py.
The metric is target-token negative log likelihood — how much probability each
GGUF assigns to the official continuation, token by token. This avoids judging
quality from a single sampled answer.
| Metric | v1 (synthetic) | v2 (real imatrix) |
|---|---|---|
| Mean NLL (lower is better) | 0.564260 | 0.522266 (−7.44%) |
| Cases won (of 100) | 38 | 62 (0 ties) |
| First-token match | 44/100 | 53/100 |
| Avg greedy longest-common-prefix | 4.770 | 5.490 tokens |
All four metrics move the same direction, which is stronger evidence than the mean alone — a mean shift can come from a couple of outliers, but a 62/38 case split with better first-token agreement and longer greedy prefixes indicates a broad improvement.
Honest caveat on strength of evidence: the 62/38 split is exact-binomial p = 0.0105 — significant at p < 0.05, but not at p < 0.01, and this is a single 100-case run. Treat it as a real and consistent improvement, not a dramatic one. v2 still loses 38 cases; it wins more of them and loses them by less.
Tested on Apple M5 Max, 128 GB unified memory:
--ctx 200000 worksThroughput figures were measured on v1. v2 has an identical tensor-type histogram, so runtime cost is unchanged — the imatrix affects which codes are chosen, not how many bits are read per weight.
# ds4 (DwarfStar) — intended runtime
./ds4 -m DeepSeek-V4-Flash-0731-Layers37-42Q4K-mixed-realimatrix-v2.gguf --ctx 200000
# As a drop-in for an existing ds4 install:
ln -sfn /path/to/this/file.gguf ds4flash.gguf
./ds4-server --ctx 200000
Not a general GGUF. Like all antirez-recipe files, this requires the ds4 engine's specific tensor layout — it will not load in a generic llama.cpp/Ollama build.
Basic reasoning (greedy, temp=0) is equivalent on math, coding, and knowledge
prompts. The 0731 finetune shows a more verbose structured "show your work"
style. Across four agentic tasks via ds4-agent (file read, CSV filtering, log
scanning, and an implement→write→run→verify loop), 0730 and 0731 were
equivalent — 0731's distinguishing behavior was more transparent self-monitoring
and error recovery rather than flawless execution. A rigorous multi-turn agent
benchmark would be needed to confirm 0731's "enhanced agentic capabilities"
claim; that probe only established parity at the basic-to-intermediate level.
dspark_* config fields. Base
inference works fully; whether ds4's MTP path exploits 0731's DSpark config is
unverified. The speculative-decoding speedup may not transfer without engine
work or a DSpark-support companion GGUF.Recipe and tooling (deepseek4-quantize, the quality-testing harness, the
mixed-precision strategy) are entirely from
antirez/ds4. This is a community quant applying
that tooling to the 0731 weights, not an official DeepSeek or antirez release.