DeepSeek-V4-Flash-0731-REAM160-180B

<!-- ream-line-status -->

[!WARNING] Line status (2026-08-12): use with caution. This first-generation REAM release passed its structural and smoke checks at publication time, but the cascaded descendants built from it (REAM128, REAM96) later showed severe multi-turn degradation, and the whole REAM line is being rebuilt with a revised methodology (single-step prune-first from the base model, longer multi-turn calibration). A REAM144 replacement targeting the same memory tier is planned; this repo may be superseded.

DeepSeek V4 Flash 0731 REAM160-180B

This measured release reduces the 43-layer routed MoE from 256 to 160 experts while preserving top-k 6 routing. It contains approximately 180.445B logical parameters.

Lineage and precision

• Source: deepseek-ai/DeepSeek-V4-Flash-0731@7872f01b1d1fe23eabc4c98b48bffcef5a386062.
• Build code revision: 7d69b8490736789d47be25e6d7ccf5a8effefa16.
• Routed expert weights: source-native E2M1 FP4, block 32, with E8M0 scales.
• Attention projections and their scales remain in the source FP8 layout and are copied byte-for-byte where unchanged.
• REAM weight accumulation is deterministic FP32; BF16 is transient compute, not the primary checkpoint format.
• MTP and DSpark tensors and metadata are absent.
• Checkpoint: 100,797,983,212 bytes; committed NATIVE_READY.json SHA-256 68fc46c987891e6e763611612262b8fae0cbd3de7de300efa169b5701bfaaaec.

REAM methodology

REAM160 is a calibration-guided, training-free expert merge: it does not apply gradient updates or add training examples to the checkpoint.

Calibration composition: 3,072 samples x 512 tokens.

A disjoint 512-sample heldout split was excluded from calibration.

At each layer, REAP saliency is the routed activation norm multiplied by router probability. The 160 highest-saliency source experts become centroids; the remaining source experts are assigned by their mean output and router-gate similarity, with group capacity 32. Each group is merged by saliency-weighted deterministic FP32 accumulation, router rows are remapped, and top-k 6 routing is preserved. Calibration therefore determines which source experts are retained and combined for the code, math, agent/tool, and Russian/Cyrillic workload mix; it is not fine-tuning.

Measured validation

• Hardware: 4x NVIDIA L40 44 GB.
• Wall time: 26.555 hours.
• Supervisor-accounted merge/finalization RunPod cost: $73.92.
• Comparative heldout NLL/PPL: not measured for this release.

Semantic suite: 100/100 cases passed; ru_dialogue=pass, code=pass, math=pass, agent_tool=pass, think=pass, no_think=pass.

Loading and runtime compatibility

The configuration and tokenizer can be inspected with Transformers without remote code:

from transformers import AutoConfig, AutoTokenizer

repo_id = "WaveCut/DeepSeek-V4-Flash-0731-REAM160-180B"
config = AutoConfig.from_pretrained(repo_id, trust_remote_code=False)
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=False)
print(config.num_hidden_layers, config.n_routed_experts, config.num_experts_per_tok)

End-to-end generation was validated with the REAM160 native expert-parallel loader on the hardware reported above. Stock AutoModelForCausalLM, vLLM, and SGLang loading is not claimed for this packed E2M1/E8M0 checkpoint; a runtime must implement the 160-expert DeepSeek V4 layout and its native FP4/FP8 storage.

Limitations

• MTP and DSpark are intentionally absent.
• Comparative heldout NLL/PPL was not measured for this release.
• Stock AutoModelForCausalLM, vLLM, and SGLang loading was not validated for the packed E2M1/E8M0 checkpoint.