DeepSeek-V4-Flash-0731-ream-150b

puwaer/DeepSeek-V4-Flash-0731-ream-150b

DeepSeek-V4-Flash-0731 with its routed experts reduced from 256 to 132 per layer by router-weighted expert activation merging (REAM), taking the checkpoint from 156 GiB to 79 GiB. All 43 layers are kept; only the expert population inside each MoE block changes. No fine-tuning, no distillation, and no gradient step of any kind — the low-saliency experts are merged into surviving centroids from calibration statistics in a single pass.

Produced with moe-compress.

Benchmarks

Difference from the base model, in points:

Metrics: GSM8K exact_match,strict-match, MATH-500 math_verify,none, HumanEval+/MBPP+ pass@1_plus. All greedy (n=1), 4096-token context, enable_thinking=false, served with SGLang.

What changed relative to the base model

One difference deserves to be read before you deploy this:

• The multi-token-prediction modules are gone. The base checkpoint carries mtp.0, mtp.1 and mtp.2; this one carries none of them. MTP-based speculative decoding is therefore unavailable. Engines that look for those weights will fall back to ordinary decoding. Nothing else references them, so standard generation is unaffected.

Everything else about the prompt and generation defaults is the base model's.

Thinking

Thinking is on by default, just like the base model.

# Python
tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=False)

# SGLang or llama-server API
{"messages": [...], "chat_template_kwargs": {"enable_thinking": false}}

• Reasoning effort: Set reasoning_effort to "low" (default), "high", or "max" (applies only in thinking mode).

Practical Notes

1. Token Budget: Thinking consumes tokens before the actual answer starts. Ensure your max_tokens is high enough to prevent mid-reasoning truncation.
2. Default Sampling: Per generation_config.json, the model samples by default (do_sample, temperature, top_p) rather than using greedy decoding.
3. Output Format: Reasoning appears inline in message.content, ending with </think>. (In llama.cpp, use --reasoning-format deepseek to isolate it into message.reasoning_content).

Where the prompt format comes from

DeepSeek does not ship a chat_template: the base model builds prompts with a Python encoder, encoding/encoding_dsv4.py, which does not survive a checkpoint conversion. Two things here address that.

encoding/ is copied verbatim from the base repository (MIT, Copyright (c) 2023 DeepSeek) and is the authority. Use it directly if you need tool calling, the internal task tokens, developer or latest_reminder messages, or multi-turn context — none of which the template implements.

chat_template.jinja is a transcription of encode_messages() for the subset that does fit a template: system, user and assistant turns, both thinking modes, and reasoning_effort. It is verified to reproduce the encoder string for string over several hundred conversations, including multi-turn exchanges with reasoning_content, consecutive user messages, and all three effort levels.

Compression recipe

moe-compress compress \
    --model deepseek-ai/DeepSeek-V4-Flash-0731 \
    --method ream --num-kept-experts 132 \
    --group-size 16 --merging logits --act-samples 8192 \
    --datasets c4,math,code --mix-ratio 0.0,0.3,0.7 \
    --num-samples 3072 --seq-len 512 \
    --streaming --stream-experts \
    --save-path DeepSeek-V4-Flash-0731-ream-150b

Calibration is the REAM paper's mixture, weighted 30% math / 70% code with no C4 (mix_ratio positions match datasets). --streaming is what lets a 156 GiB checkpoint be compressed on a single 96 GB GPU: layers are read and written one at a time, so peak memory is the skeleton plus one layer.

Reconstruction quality

Measured during compression on a 4096-token probe, comparing each rebuilt MoE block's output against the original, averaged over all 43 layers:

This is a compression-time diagnostic, not a quality metric — read the benchmarks for that.

Serving

Verified with SGLang. On Hopper the MXFP4 expert layout needs an explicit MoE runner; auto lands on a Triton path that asserts on the packed weights:

python3 -m sglang.launch_server \
    --model-path puwaer/DeepSeek-V4-Flash-0731-ream-150b \
    --tp-size 2 \
    --nnodes 2 --node-rank $RANK --dist-init-addr <head>:5000 \
    --moe-runner-backend flashinfer_mxfp4 \
    --chat-template chat_template.jinja \
    --context-length 4096

GGUF builds for llama.cpp are available; the routed experts are already MXFP4 in the source weights, so an MXFP4_MOE GGUF is numerically identical to this checkpoint.

Choosing between REAP and REAM

On this model REAP wins outright, and by a margin that widens as more is removed. Points against the base model, given as 178 experts / 132 experts:

REAP at 178 experts returns the base model's pass@1 exactly on both code benchmarks — 542 problems, not one of them different — for a third off the checkpoint.

An earlier build of these checkpoints looked like a trade-off instead — REAP holding arithmetic and losing code, REAM the reverse. That was an artifact of a bug in the hash-routed layers' expert table, not a property of either method. With it fixed the trade-off disappears: REAP is at least as good as REAM on code and far better on arithmetic.

Note also how differently the two scale. Going from 178 experts to 132 costs REAP 1.1 more points of GSM8K; it costs REAM 17.0.

Citation

The methods:

• REAP — Router-weighted Expert Activation Pruning. Lasby et al., 2025. arXiv:2510.13999
• REAM — Router-weighted Expert Activation Merging. Jha et al., 2026. arXiv:2604.04356

The implementation: https://github.com/puwaer/moe-expert-compress

The base model: deepseek-ai/DeepSeek-V4-Flash-0731

License

MIT, following the base model. The compression code is MIT; its numerical core is ported from the official REAM reference implementation (Copyright (c) 2026 Samsung Electronics Co., Ltd.) with attribution headers retained. encoding/encoding_dsv4.py is DeepSeek's, redistributed under the same MIT terms as the base model.