glm-5.3-flash-tiny-random-nvfp4

GLM-5.3 Flash tiny-random NVFP4 fixture

This is a deterministic, randomly initialized test checkpoint derived from the configuration and tensor schema of LibertAIDAI/GLM-5.3-Flash-NVFP4, pinned at revision 9e0d74e3cef17f634e84fb8e2223707e02616290.

It is intended to reproduce vLLM loader, cache-layout, sparse-attention, NVFP4 MoE, and MTP compatibility problems on GB10/SM121 without loading the full checkpoint. It is not useful for language or vision tasks, and its output is deliberately meaningless.

The companion GB10 runtime and source patch live at cyijun/glm-5.3-flash-nvfp4-gb10.

What remains architecture-faithful

The fixture keeps dimensions that select execution branches, kernels, packed weight layouts, or cache ABIs:

The routed-expert tensors use the same ModelOpt NVFP4 storage ABI as the source checkpoint: packed U8 weights, F8_E4M3 per-block scales, and scalar F32 global scales. All 1,728 expert projections are present.

To keep the artifact under 1 GiB, hidden_size is reduced from 4096 to 256, the 45-layer stack is reduced to its first four-layer cycle, maximum positions are reduced to 8192, and the vision tower is reduced to one small layer. The four base layers plus one copied MTP layer still cover KDA, dense MLP, sparse MLA, routed NVFP4 MoE, shared experts, and speculative decoding.

What the fixture exposed on GB10

The pinned base image vllm/vllm-openai:glm53-flash-arm64-cu130@sha256:905c02933be6021301db2dc284e24e3727467aa3a0f63b41d609885778a07bce loads the checkpoint but needs three compatibility changes:

1. GLM-5.3 is logically NoPE (qk_rope_head_dim=0), while the SM120 fp8_ds_mla cache writer requires a physical 64-BF16 positional region. The adapter writes zeros into that region and pads the absorbed 512-wide query to 576. This preserves the NoPE dot product.
2. index_topk=2048, index_kpool=4, the always-selected tail, and BLOCK_N=128 alignment produce a physical sparse-index buffer of 2176 columns. The bundled FlashInfer AOT module only instantiates 2048. The adapter adds the exact num_heads=64, topk=2176 decode specialization.
3. Requests with more than 64 query tokens use the prefill orchestrator, so the matching GLM_NSA, num_heads=64, topk=2176 prefill specialization is also required. The patched module is rebuilt for sm_121a; editing Python or JIT source alone does not override the precompiled AOT .so in the base image.

The physical sparse-MLA row remains the native GLM_NSA 656-byte layout:

query = [absorbed NoPE 512 | zero padding 64]
KV    = [FP8 latent 512 | four FP32 scales | 64 BF16 zeros]

The vLLM CLI still requires --block-size 256 because the GLM index cache must be divisible by index_kpool * 32. Internally the compressed FlashInfer sparse cache uses 64-token physical pages; these are different levels of the cache layout and should not be conflated.

Run with the GB10 image

docker run --rm --gpus all --ipc=host -p 8000:8000 \
  ghcr.io/cyijun/glm-5.3-flash-nvfp4-gb10:vllm-glm53-sm121 \
  cyijun2k/glm-5.3-flash-tiny-random-nvfp4 \
  --served-model-name glm53-tiny \
  --max-model-len 512 \
  --max-num-seqs 2 \
  --gpu-memory-utilization 0.10 \
  --block-size 256 \
  --moe-backend marlin \
  --enforce-eager

Optional one-token MTP smoke test:

--speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Validation evidence

Validation was performed on an NVIDIA GB10 (SM121) with vLLM 0.1.dev20051+g487ecf187, FlashInfer 0.6.17, Transformers 5.15.1, and PyTorch 2.13.0+cu130.

See VALIDATION.json for the exact image digest and machine-readable results.

Integrity

model.safetensors is 1,010,842,392 bytes and contains 5,342 tensors.

sha256 d0f8a670c6d96be8984db205a4bfb32c947f09182d6b327fab8d4f1088b66810

The checkpoint is deterministic with model seed 42 and expert payload seed 4242. BUILD_INFO.json records the source revision and fixture profile.