GLM-5.3-W4AFP8

GLM-5.3-W4AFP8

W4AFP8 quantization of zai-org/GLM-5.3, quantized from the BF16 master: MoE expert weights in INT4 (group-128, AWQ-calibrated), activations and non-expert layers in FP8, served by SGLang's native CUTLASS W4A8 grouped GEMM on Hopper (SM90). Roughly 2x the KV-cache capacity of the FP8 checkpoint at identical serving throughput, with EAGLE/MTP speculative decoding intact.

Also mirrored on ModelScope (魔搭) for faster downloads in China.

Successor to PhalaCloud/GLM-5.2-W4AFP8; GLM-5.3 is architecturally identical to GLM-5.2 (verified tensor-by-tensor), so the same serving configuration applies.

What is different from the 5.2 checkpoint

• Quantized from the BF16 master (zai-org/GLM-5.3-BF16), not the FP8 release, avoiding double quantization of expert weights.
• AWQ calibration on coding-agent traces (SALT-NLP/SWE-chat, 128 windows x 2048 tokens, session-disjoint split), matching the production agentic workload, instead of generic web text.
• The MTP/nextn draft layer (layer 78) ships quantized; EAGLE speculative decoding works out of the box.

Evaluation

Measured on this checkpoint served with the SGLang configuration below (official request parameters, reasoning at the chat-template default).

References: Artificial Analysis GLM-5.3. Per-item artifacts are preserved in the repository history.

Serving

python -m sglang.launch_server \
  --model-path PhalaCloud/GLM-5.3-W4AFP8 \
  --tp 8 \
  --quantization w4afp8 \
  --disable-shared-experts-fusion \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 \
  --reasoning-parser glm45 --tool-call-parser glm47

H200-class GPUs serve the full 1M context; on H100 80GB use --mem-fraction-static 0.78 and a reduced context length. vLLM support for this exact checkpoint format is pending upstream in vllm PR #53848 (--quantization w4afp8 via the humming backend).

Verification

• Structural audit: tensor inventory (176,997 tensors) matches the format-proven GLM-5.2-W4AFP8 checkpoint exactly (names, dtypes, shapes).
• Dequantization audit: int4 expert weights reconstruct within the expected group-128 envelope (10.8-13% relative RMS residual after AWQ channel-scale fitting; no outliers).
• Held-out teacher-forced NLL and task benchmarks: see Evaluation above.

The per-layer AWQ search checkpoints (scales and clip thresholds for all 78 layers) are preserved in the repository history for reproducibility and further research; they are kept out of the current revision so snapshot_download fetches only the serving weights.