DeepSeek-V4-Flash-0731-REAM96-111B

DeepSeek V4 Flash — REAM96 (111B)

A slimmed-down DeepSeek-V4-Flash-0731: 96 of the original 256 experts per layer, picked and carried over byte-for-byte. Same 43 layers, same shared expert, same top-6 routing — just a smaller crew. The 2-bit build of this checkpoint targets 48 GiB machines; this repo is the full-precision source of that build.

[!TIP] The method is a REAP × REAM hybrid — not purely either. Experts are pruned REAP-style: ranked by how much they actually contribute on real traffic, with the survivors copied over byte-for-byte, untouched. But the keep-list isn't a plain top-N — every domain (Russian, code, tool use, math…) gets a protected quota of its own specialists. And the few experts that turned out to be near-duplicates were merged REAM-style instead of dropped. The router is then re-balanced so the smaller crew is used the way the original was. All of it in one step from the original model — no cascades.

[!IMPORTANT] This checkpoint keeps the original packed FP4/FP8 weight layout and the custom 0731 architecture. It is not loadable with stock transformers generation, vLLM or SGLang — it exists for the DS4 fork toolchain and for making quantized builds. Want something you can just run? Grab the ready 2-bit build: DeepSeek-V4-Flash-0731-REAM96-111B-DS4-GGUF.

[!WARNING] Live smoke testing passed 5/10 scenarios on the first run. Independent reruns show the failures (Russian wordplay, multi-turn, English → Russian code-switching, Tool calling (DSML), Long-dialog focus (drift check), Tool call → code chain) are intermittent, not absolute — see the Stability column below for per-scenario pass rates. Multilingual chat, reasoning and long dialogs are consistently healthy.

What's in the repo

≈70 GB of sharded Safetensors plus config and tokenizer. Everything follows the original 0731 format, so tooling that understands the base model understands this one. The config and tokenizer load fine with AutoConfig / AutoTokenizer; per-layer pruning choices are recorded in SELECTION.json.

DSpark

The model's built-in speculative decoder (three extra MoE stages that draft tokens ahead) is preserved untouched under its mtp.* tensor namespace. In the GGUF release it ships as a separate optional file — see the companion repo.

How it was made

One pruning step, straight from the original — no cascading. Expert importance was measured by running deepseek-ai/DeepSeek-V4-Flash-0731 over a ~5-million-token calibration mix (multi-turn dialogs, thinking and direct modes, rendered with the model's own chat encoder). The strongest experts of every domain were protected from pruning, the survivors were carried over byte-identical, and the router was re-balanced to keep the original selection behavior.

This line replaces the earlier cascaded REAM builds (now archived under -exp names), which degraded badly in multi-turn use.

Smoke results

Every scenario is a live multi-turn conversation, run on the companion 2-bit GGUF build — the only runnable form of this checkpoint. Treat the results as a lower bound for this full-precision source (raw evidence ships in the companion repo's SMOKE_REPORT.json).

Stability = pass rate over independent reruns of the scenarios that failed the first run; passing scenarios were not re-run.

Limitations

• Runs only through the DS4-fork ecosystem; this repo is the archival/source form.
• Pruning is training-free: rarely-used specialist skills of the removed experts are gone by design. See the smoke table for what was verified.