GLM-5.2-REAP-NU176-526B

GLM-5.2-REAP-NU176-526B — non-uniformly expert-pruned GLM-5.2 (NVFP4)

The first non-uniform REAP cut of GLM-5.2: instead of pruning every MoE layer to the same expert count, each layer keeps a different number of routed experts (108–216, average 176 of 256) allocated by per-layer REAP saliency measured over 16.08B calibration tokens. ~526B params, 337GB on disk, NVFP4 routed experts.

Why non-uniform

Uniform keep-K pruning drops 16–26% of per-layer routing mass depending on the layer — layers 7–17 of GLM-5.2 are hurt far more than layers 3–6 or 77. This cut reallocates the expert budget to where the router actually concentrates saliency:

Mean dropped saliency mass: 17.6% vs 22.3% for the prior uniform-156 cut.

⚠️ Requires a (tiny) vLLM fork

Stock vLLM assumes one expert count for all layers. This checkpoint carries config.num_routed_experts_per_layer (list of 79 ints; n_routed_experts = max = 216) and each layer's experts are renumbered 0..k−1 with matching gate.weight / e_score_correction_bias rows.

The required patch is 12 lines in deepseek_v2.py (DeepseekV2MoE resolves its layer's count from the list; everything else — weight loading, MTP, ModelOpt NVFP4, CUDA graphs, TP, DCP — works unmodified because the scalar stays at the max).

One-command serving (docker, 4× RTX PRO 6000 Blackwell SM120 or similar 4×96GB): see https://github.com/0xSero/glm-5.2-nonuniform — 262k context @ concurrency 2–3 (DCP4, FP8 KV, MTP speculative decoding; ~40 tok/s at DCP4/262k, ~77 tok/s at DCP2/131k).

Prebuilt forked-vLLM image (pick either):

• GHCR: docker pull ghcr.io/0xsero/vllm-b12x-nonuniform:20260705
• This repo: docker/vllm-b12x-nonuniform-20260705.docker.tar.zst (10.6GB) — zstd -dc *.docker.tar.zst | docker load

How it was made

1. Per-(layer, expert) REAP saliency (gate-weight × expert-output norm) merged from 5 calibration runs (GLM-5.2 + GLM-5/5.1 cross-checks), 16.08B tokens.
2. Per-layer keep counts via water-filling on layer-normalized saliency curves (budget = 75×176; counts clamped to [108, 216], multiples of 4).
3. Whole-expert copy from nvidia/GLM-5.2-NVFP4 with per-layer renumbering, router rows sliced; MTP layer experts taken from the NVFP4 uniform cut (NVIDIA ships MTP in BF16, which b12x-class MoE kernels reject).
4. Anti-overthinking serving per arXiv:2606.00206 (quantized/pruned reasoning models overthink; a hesitation-marker penalty/directive restores CoT efficiency).

Observation data: 0xSero/glm-5.2-reap-observations, 0xSero/glm52-reap-traces.

Lineage

• Base: nvidia/GLM-5.2-NVFP4 (753B-A40B, 78L, 256E, MLA+DSA, MTP)
• Method: REAP (Router-weighted Expert Activation Pruning), extended to per-layer budgets
• Prior uniform cut: 0xSero/GLM-5.2-504B (keep-168)