DeepSeek-V4-Flash-0731-REAP-DSpark

DeepSeek-V4-Flash-0731-REAP + DSpark MTP draft head — complete checkpoint

A drop-in replacement for 0xSero/DeepSeek-V4-Flash-0731-REAP with a fine-tuned DSpark MTP draft head merged in. Download it and you have the whole model: no patching step, no separate head file to graft on.

Every emitted token is bit-identical to base autoregressive decode. The speed-up is lossless by construction — speculation changes how tokens are produced, never which tokens they are — and that invariant is checked on every run by a gate that compares the speculative path against plain AR.

What is actually different from the base checkpoint

Three files. Shards 46, 47 and 48 carry the MTP draft-head tensors; shards 1–45, the tokenizer, and the config are byte-identical to the base. If you already have the base checkpoint, the only new weights here are ~6.7 GB of head shards.

Measured speculative decode

Measured on a Jetson AGX Thor (sm_110a, 122.8 GiB unified) with a from-scratch pure-CUDA inference server. Every figure is from a logged run, not an estimate.

Acceptance by workload, and where this head gives ground

tau is tokens committed per target forward. The head was selected on the mean, and the mean hides a real trade — published here rather than buried:

The gain is concentrated in the constructive categories — reasoning nearly doubles — and it is paid for on the two reconstructive ones the stock head was already best at. Under this project's own release rule that is a failure of 3 of 6 category floors, and the rule is stated so that a downstream user can predict their own workload instead of inheriting a mean.

If your workload is dominated by long-context reconstruction, the stock head may serve you better. For mixed agentic work — tool calls, multi-turn, code edits, reasoning — this head is a clear win.

Task accuracy

Measured through the same server, effort=low, 24k token budget, temperature 1.0 / top-p 0.95.

A caveat that matters more than any single number. GPQA at a 24k budget was pre-registered at 87–93 % before the run, with a stated falsification threshold of 85 %. It landed at 83.8 % — the prediction was wrong, and the reason is a finding about this pruned model: 19 of 51 items that exhaust an 8k budget never terminate even at 24k, and the ones that do finish score 78.1 %, not the 93.9 % that items terminating inside 8k reach. Prompts needing long chains are qualitatively harder here, not merely longer. Budget your context accordingly.

Context

KV packing is bit-exact (720 B/row vs 2048; the fp32 path already stored e4m3-representable values in four bytes) and costs ~12 % of prefill throughput, so it is worth enabling only when the context needs it.

Limitations, stated plainly

• Measured on one machine. Every number above comes from a single Jetson AGX Thor. Throughput will differ on other hardware; accuracy should not.
• Single-stream. The server these numbers came from serves one request at a time by design. This is not a multi-tenant serving checkpoint.
• The eval battery is partial. The suite is complete; the 24k extension covers 4 of 8 tasks. LiveCodeBench, MATH-500, HumanEval and SciCode are at the 8k budget and several are truncation- limited, which is why LCB is marked not quotable above.
• This is a REAP-pruned model. 160 of 256 experts per layer. Comparisons against unpruned DeepSeek-V4-Flash are not like-for-like.

Licence

MIT, following the base checkpoint. The DeepSeek copyright notice is retained in LICENSE. This repository adds only the fine-tuned draft-head tensors in shards 46–48.