GLM-5.3-Flash-MXFP4

OneNexus/GLM-5.3-Flash-MXFP4

An MXFP4 GLM-5.3-Flash checkpoint for text and image-to-text serving on AMD Instinct MI355X (gfx950). On matched text-reasoning samples against amd/GLM-5.3-Flash-Quark-MXFP4, it scores +1.0 percentage point on MMLU and +3.5 points on GPQA Diamond, with −0.2 point on GSM8K. These are observed differences on fixed samples, not statistically established superiority or a claim about vision, safety, or general agentic quality.

Runtime support: SGLang PR #39273.

Model at a glance

Text-reasoning quality

Both checkpoints answered the same fixed questions with one greedy attempt per question. A capped or unfinished response counts as incorrect.

MMLU and GPQA results are sensitive to the answer-length limit: MMLU has 43/500 truncated outputs for each checkpoint, and GPQA has 55/198 for OneNexus versus 61/198 for AMD. The paired exact-test p-values are 0.458 for MMLU and 0.265 for GPQA; neither difference reaches p < 0.05. The samples neither prove superiority nor equivalence. See the evaluation protocol and per-question aggregate data for settings, truncations, intervals, and paired discordances. These are fixed-subset text-reasoning evaluations, not full benchmark leaderboard scores.

Serving performance against AMD

The checkpoints were replayed on the same four MI355X VFs with the same SGLang build, tokenizer and prompt rendering, TP4/EP4 placement, FP8 KV cache, EAGLE 5/1/6 settings, cache policy, 393-entry AgentX corpus, and 900-second measurement per concurrency level. Each checkpoint used its own declared target and MTP weight precision.

Across c1–c48, OneNexus is 0.1% lower in geometric-mean aggregate output throughput and 1.0% higher in geometric-mean P90 per-user output rate. This is a measured performance-parity result, not a throughput win. Both runs have zero request errors (9,223 OneNexus and 9,264 AMD requests). P90 tok/s/user is the 90th percentile of each eligible request's reciprocal mean inter-token latency, not aggregate throughput. The OneNexus and AMD summaries contain the full rung-level evidence.

How the serving path works

Sparse attention and K-pool selection

GLM-5.3-Flash uses 256-wide sparse-attention projections with no RoPE tail. The separate K-pool index cache stores 128 FP8 index-key values and one FP32 scale (132 bytes per row), distinct from the 512-wide MLA attention cache. The native gfx950 path reads and writes the same preshuffled index layout. It also keeps selection bounded at long context without dropping valid positions: 512 selected four-token groups cover up to 2,048 history tokens, and up to three recent unpooled tokens bring the mapped width to 2,051.

Verification compute and EAGLE

Target verification runs AITER MXFP4 experts, supported gfx950 mHC, and shape-scoped fused KDA, with reference fallbacks where required. Expert execution retains the model's SwiGLU gate and clamp semantics. Five steps of the in-checkpoint MTP head propose tokens, then one target pass verifies them. Graph replay reduces repeated launch work; KV and Mamba state must still refer to the same visible committed prefix after rejection or stop-token truncation. SGLang PR #39273 contains the implementation and correctness tests.

On the c16 AgentX rung, acceptance length is 4.9072 and acceptance rate is 78.14% over 1,595 completed requests, as recorded in the serving results.

Measured optimization ladder: from KDA traces to kernel selection

The c16 serving capture puts the kernel work in context: target verification contains 535.5 ms of MoE, 112.6 ms of mHC, and 91.4 ms of KDA device-event residence per rank, averaged across four ranks. MoE is the largest of these; KDA is a smaller, concrete opportunity to reduce repeated launches and work.

The following isolated MI350X VF / gfx950 operator study shows how the KDA implementation was tuned. It uses synthetic GLM TP4-shaped tensors, not a checkpoint comparison or a new MI355X serving run. The first example has eight requests and eight verification tokens per request.

1. Observe two kernels. The reference graph launches causal convolution, then gating and delta-rule recurrence. Its paired operator time is about 38.9 μs.
2. Fuse, then inspect the result. The initial fused kernel removes the intermediate launch, but a four-value tile (BV=4) recomputes the Q/K convolution for each of 32 value tiles. Its trace has one kernel per replay, yet paired time rises to 41.7 μs. Launch count alone is not sufficient evidence of an improvement.
3. Reduce duplicated work. A 16-value tile (BV=16) shares that work across more values, reducing the value-tile grid from 32 to 8. The trace still has one kernel per replay, now visibly shorter; separately measured graph time falls to 28.4 μs, 27.0% less time than the reference.

The timelines retain actual event durations and gaps from the first three of eight captured replays. The timing bars come from a separate, unprofiled graph microbenchmark: three reference/fused/fused/reference (ABBA) repeats, with fixture clones excluded. They are operator measurements, not a percentage to add to the AgentX throughput curve.

Check the serving shape and keep the fallback

EAGLE 5/1/6 uses six-token verification. The matched six-token sweep below confirms the direction at seven batch sizes: 31.50 → 23.02 μs at batch 8 and 37.66 → 33.03 μs at batch 16. The smaller batch-16 gain shows why dispatch must depend on shape. Batch 1–2 retain the four-value tile; the qualified larger batches use the 16-value tile.

The PR reuses the existing fused KDA path only for gfx950 batches 1–16, six or eight verification tokens, 16 local heads, 128-wide states, BF16 activations, FP32 convolution weights, no convolution bias, and the −5 gate bound. Other ROCm shapes retain the reference path. The same study checked three changed-input graph replays for every measured shape: active output and convolution buffers matched exactly, while intermediate SSM states passed an absolute tolerance of 0.004.

The measurement data and kernel-event excerpts retain every ABBA timing, including slower tile choices. The study protocol pins hardware, source hashes, the trace extraction, and the relationship to the PR kernel. This is a measured kernel-tuning ladder; the complete checkpoint-level FP8/MXFP4 and B300-estimate comparisons follow below.

Additional measured tuning choices

mHC backend selection. GLM-5.3-Flash mixes four residual streams across 4,096 channels around its attention and MLP blocks. With AITER enabled on gfx950, the runtime selects its HIP kernels for this geometry. The saved 32-token traces show shorter projection/squared-sum, mixing, and post kernels, with the same launch count and caller RMSNorm. Paired graph time falls from 30.60 to 10.01 μs for pre plus RMSNorm and 9.66 to 2.03 μs for post. Both routes pass the same numerical oracle.

Vision merge as a matrix multiply. The downsample has kernel size equal to stride (2×2), no padding, and one group: each patch is an independent linear projection. The existing Conv2dLayer uses unfold + linear for that case, bypassing MIOpen convolution search. On the model's noncontiguous [256, 1024, 2, 2] merge layout, the warmed trace replaces the transpose/convolution/add sequence with a layout copy and a bias-enabled matrix multiply. Fresh-process first-call time falls from 4.501 to 3.307 seconds; warmed graph time falls from 63.28 to 25.62 μs. BF16 output passes the same numerical threshold.

These are isolated operator measurements, not percentages attributed to the AgentX serving curves. The study protocol and raw traces and machine-readable results pin the environment, repeats, errors, and source revision.

Serving frontier versus supplied B300 estimates

The green MI355X curve below is measured on four gfx950 VFs with this checkpoint. The blue B300 curve consists of supplied estimates for four B300 GPUs; we did not reproduce B300 runs or obtain their raw artifacts. The disclosed corpus, 256K context cap, GPU count, and EAGLE shape align, but hardware, DSA backend, and serving installations differ. This is a frontier illustration, not a controlled cross-hardware speedup claim.

At c48, the measured MI355X arm has 36.9% higher aggregate output rate but 19.1% lower P90 per-user rate than the supplied B300 estimate. The measured MI355X c64 point is dominated by its c48 point, so it is shown as a diagnostic in the figure but not a frontier row or claim. Across eight MI355X rungs, 11,810 requests completed with zero errors. The measured results, supplied B300 values, and figure renderer are available.

Serving uplift versus the official FP8 checkpoint

Both curves here were measured with SGLang source 5fd4387a753cdc88c0c87efb1bcabab1efabfc93 on the same image, GPU set, tokenizer, TP4/EP4 placement, FP8 KV policy, EAGLE 5/1/6 shape, graph limit, cache reset, 393-entry AgentX corpus, and 900-second rungs. The complete checkpoints and their declared MTP precision differ; this does not isolate quantization-kernel speed. This is a separate matched replay from the AMD comparison above, so its OneNexus rates differ slightly.

Over c1–c48, MXFP4 gives +12.4% geometric-mean aggregate output rate and +18.2% geometric-mean P90 per-user rate. Both arms finished with zero request errors (9,734 FP8; 11,810 MXFP4 across all eight measured rungs). Indexed tensor payloads are 305.78 GiB for FP8 and 182.12 GiB for MXFP4, a 40.44% checkpoint-file reduction, not a peak-memory comparison. With the same memory-allocation policy, measured total weights use 77.69 versus 46.99 GB/GPU and device KV capacity rises from 13.97M to 17.32M tokens. The FP8 and MXFP4 summaries, matched protocol, and figure renderer contain the full per-rung evidence. Dominated c64 observations are visible in the figure for context but excluded from the reported frontier uplift.

Reproduction

The serving recipe specifies the checkpoint revision, container, source, tokenizer, and launch arguments. The AgentX measurements use SGLang source 5fd4387a753cdc88c0c87efb1bcabab1efabfc93; the recipe and qualification record identify the source used for serving qualification separately. Quality runs use BF16 KV with graphs and speculation disabled; see the evaluation protocol for the evaluator, inputs, and request policy. Raw restricted prompts are not redistributed.

License

MIT, with source attribution in NOTICE.