Qwen3.8-Flash-Next-REAP-384-bf16

Qwen3.8-Flash-Next REAP-384 (bf16)

This is the full-precision REAP-384 checkpoint. It is the 384-of-512 expert build spliced directly from the original bf16 weights, at native precision, before any quantization. Every other REAP-384 artifact we publish, the MLX 4-bit, any GGUF or AWQ conversion, derives from these bytes. This checkpoint exists for the cases where you need the full-precision source: requantizing to a format of your choice, fine-tuning, or serving at maximum fidelity.

REAP-384 keeps more experts than REAP-288 (384 of 512 vs 288 of 512), so it is the closest-to-stock member of the family: the 4-bit build scores 92.1% on HumanEval, against 93.9% for the unpruned 512-expert conversion and 91.5% for the smaller REAP-288. It is the quality-leaning pick when your disk budget allows 80 GB; REAP-288 is the quality-per-GB pick at 68 GB.

You do not need pmlx to use this model. The runnable REAP-384 build loads and serves on stock mlx-vlm today, with no patches and no extra engine. omlx v0.6.4+ also runs the MLX builds today and streams the n-gram table to a lower footprint. pmlx is a separate inference engine that is not public yet; it releases alongside the tiered model and makes the same MLX builds faster, but everything you need to run REAP-384 today works without it.

If you just want to run the model

Use the 4-bit build: sh0wie/Qwen3.8-Flash-Next-REAP-384-MLX-4bit. It loads on stock mlx-vlm with no patches at ~80 GB resident, drops to ~51 GB when a streaming runtime keeps the n-gram table on NVMe (omlx v0.6.4+ does this today), and scores 92.1% on HumanEval. It is roughly a quarter of this checkpoint's size and its quality is within two points of native precision. For most uses, that is the model you want.

pip install git+https://github.com/Blaizzy/mlx-vlm.git
python -m mlx_vlm.server \
  --model sh0wie/Qwen3.8-Flash-Next-REAP-384-MLX-4bit --port 8080

Decode-speed figures for the 4-bit build are on that build's own card. This bf16 checkpoint is the quality-and-source tier; because it pages far more bytes per token than the 4-bit build, we do not quote a tok/s figure for it here.

About this build

• 180B-parameter class: 125B main model, 51B n-gram embedding table, 48 layers alternating Gated DeltaNet and Qwen sparse attention, each with a 384-expert MoE routing top-10.
• 384 of the original 512 experts per MoE layer, chosen by REAP saliency calibrated on the quantized weights over ~686K tokens of agentic-coding traffic. The kept-expert set is identical to the 4-bit build.
• Native bf16. No quantization error is baked in.

What the bf16 source is for

• Requantizing. This is the reference source. Convert it to MLX at any bit width, to GGUF for llama.cpp, to AWQ or an FP variant, without inheriting a prior quantization's rounding. Start from bf16 and you lose accuracy exactly once.
• Fine-tuning. LoRA or full fine-tunes want full-precision weights. The 4-bit build is for inference; this is what you train on top of.
• Max-quality serving. If you have the memory budget and want the highest fidelity the pruned model can produce, serve these weights directly. At 294 GB that means paging on a single Mac; the details are below.

<details> <summary><strong>Running the bf16 source on the pmlx engine (coming with the tiered release)</strong></summary> <br>

pmlx is a pure-MLX inference engine for Apple Silicon. It is not public yet; it releases alongside the tiered model, and the install URL below goes live then. None of this is needed to run the 4-bit build on mlx-vlm today. It is here because pmlx is what lets you serve the 294 GB bf16 source on a single Mac at all, by keeping part of the model resident and paging the rest from SSD.

# available when pmlx ships with the tiered release
pip install git+https://github.com/gethamster/pmlx
python -m pmlx.server \
  --model sh0wie/Qwen3.8-Flash-Next-REAP-384-bf16 --port 8123

The server is OpenAI-compatible (/v1/chat/completions, SSE streaming).

Residency is automatic. On load, pmlx reads how much memory your Mac has, reserves room for the KV cache (which grows with context length), and keeps as many experts resident on the GPU as fit; the rest page in from SSD as they are needed. On a 294 GB model that paging is how the model runs at all, and pmlx sets the resident fraction to whatever your machine can hold. The larger n-gram table streams from NVMe rather than sitting in RAM. Nothing is baked into the checkpoint, so the same download behaves correctly on differently-sized Macs without a re-download.

Pinning the ratio. When you want to set the resident fraction yourself, --resident-ratio does it:

python -m pmlx.server --model sh0wie/Qwen3.8-Flash-Next-REAP-384-bf16 \
  --resident-ratio 0.40 --port 8123

Resident memory follows one relation: a fixed backbone, plus room for the KV cache, plus the share of expert weights you keep on the GPU.

resident ~= backbone + KV_reserve + resident_ratio * expert_weight

The n-gram table is not in that sum; pmlx reads it from NVMe a few hundred bytes at a time, bit-identical to holding it in memory. So the only term you scale is the expert share. REAP-384 carries more experts than REAP-288, so at the same ratio it keeps proportionally more expert weight resident. Keep 40% of the experts and you carry roughly 40% of the expert weight; a higher ratio keeps more experts hot and costs more memory, a lower ratio does the reverse. Quality does not move with the dial, because paged, cached, and prefetched expert gathers are bit-identical to the fully-resident path.

Serving bf16 pages far more bytes per token than the 4-bit build does, so throughput is lower and we do not quote a tok/s figure for it here. Treat this checkpoint as the quality-and-source tier, and the 4-bit build as the one you reach for when you want speed.

</details>

Provenance

• Qwen/Qwen3.8-Flash-Next: upstream weights.
• This build: REAP expert pruning 512 -> 384 per layer, calibrated on-device over ~686K tokens of agentic-coding traffic, then materialized at native bf16. The per-layer kept-expert manifest is the same selection shipped as reap_kept_experts.json in the 4-bit build.

Limitations

• Calibration reflects one team's agentic-coding distribution. Retention numbers should not be read as general-domain; domains far from code may degrade more.
• This is a source checkpoint, not a convenience build. At 294 GB it does not fit in RAM on a single Mac; serving it directly needs a paging engine such as pmlx (see above) or your own large-model stack. To simply run the model, take the 4-bit build, which loads on stock mlx-vlm.
• Single-run evaluations on the lineage, no confidence intervals. Vision input is untested after pruning.

License

Qwen Community License 1.0, inherited from the base model Qwen/Qwen3.8-Flash-Next.