DeepSeek-V4-Flash-moet2pf

DeepSeek-V4-Flash — 2-bit expert plane file (.moet2pf) for vLLM-Moet

One file that puts DeepSeek-V4-Flash (159B) on a single NVIDIA DGX Spark. This is the offline-built 2-bit plane store for vLLM-Moet: all 11,008 routed experts (43 MoE layers × 256) of deepseek-ai/DeepSeek-V4-Flash, requantized to sign-symmetric {−4,−1,1,4} 2-bit planes and serialized per-expert-slot into one O_DIRECT-servable file. The serving stack reads experts straight from this file into a GPU cache pool — the checkpoint's 148.66 GiB of expert tensors are never materialized in RAM, which is what makes a 119.7 GiB unified-memory box workable.

This file does not replace the checkpoint — it pairs with it. At boot vLLM still reads the upstream checkpoint for the dense/attention/gate/MTP weights and tokenizer; this file serves only the routed experts. You need both.

Measured (DGX Spark / GB10, fully resident, deterministic config)

16.6 tok/s decode (29.8 with MTP k=2) · prefill ~413–665 tok/s · 64K window · greedy byte-identical 3/3 · needle 8/8 · tool + reasoning parsing on both the OpenAI and Anthropic API surfaces. Full methodology and boot discipline: docs/dgx-spark.md.

Use

# 1. this file + the upstream checkpoint
hf download 9prodhi/DeepSeek-V4-Flash-moet2pf ds4-planes.moet2pf --local-dir /data/models
hf download deepseek-ai/DeepSeek-V4-Flash \
  --revision 60d8d70770c6776ff598c94bb586a859a38244f1 \
  --local-dir /data/models/DeepSeek-V4-Flash

# 2. serve (image build + all knobs: docs/dgx-spark.md in the GitHub repo)
git clone https://github.com/9prodhi/vLLM-Moet && cd vLLM-Moet
DOCKER_BUILDKIT=1 docker build -f Dockerfile.gb10-v024 -t vllm-moet-gb10:v024 .
docker run -d --name vllm-moet --gpus all --network host --ipc host \
  -v /data/models/DeepSeek-V4-Flash:/workspace/models/DeepSeek-V4-Flash:ro \
  -v /data/models/ds4-planes.moet2pf:/workspace/models/ds4-planes.moet2pf:ro \
  vllm-moet-gb10:v024

Boot only through the image's guarded entrypoint (it preflights the unified-memory transient and refuses unsafe boots — a raw vllm serve can freeze a Spark).

Verify

Every slot carries a sha256; the serving stack re-verifies the file at each boot (~12–15 s O_DIRECT). Offline:

python3 tools/build_plane_file.py verify --dst /data/models/ds4-planes.moet2pf
# -> OK: 11008 slots verified (a single flipped body byte fails loudly, no SIGBUS)

Whole-file checksum: sha256:998610cd65bbdad730229492dc2e3da2a1723ef73529fb027e676475c70ead35

Reproduce instead of downloading

The file is deterministic output of tools/build_plane_file.py over the pinned checkpoint (~7 min on the Spark GPU, peak RSS < 1 GiB; slot bytes are byte-identical to what the loader would stage in memory — checked by a golden sample at build time). See docs/dgx-spark.md §3, Option B.