GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED-GGUF

GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED-GGUF

The balanced GLM-5.3-Flash quant for AMD Strix Halo (Ryzen AI Max+ 395 / gfx1151). Measured, not marketed: loads on ONE Strix Halo box with zero special accommodations — the smallest file (71.8 GiB) and the no-fuss daily driver.

Brand: PaoAI · community quant, not affiliated with Z.AI.

---

Updated 2026-09-21 — re-measured on the latest version (paoai-strix-engine, strix-main @ 4a8440f58): decode faster at every depth (+32% at 128K), chain re-run stated honestly below, all sweep points integrity-checked. Weights unchanged.

What is this? (30 seconds)

We took the 642 GB GLM-5.3-Flash and compressed it organ by organ: the parts that need precision keep it, the parts that don't get compressed harder. The BALANCED mix stays small enough for one box with no offload flags.

The recipe (what's inside)

Chain-test results (N=3 medians — Sep 2026)

We run a fixed coding exam ("chain test") where tasks are graded by execution, not opinion: the model reads deceptive code and must trace it, implements a novel state machine, and writes a thread-safe ring buffer that survives a 5,000-operation stress harness. Three identical runs per model; we report medians with spreads (identical runs vary — single scores are unreliable).

Task median: 67/100 (runs 60 · 73 · 67)

Re-run 2026-09-21 on the latest version (4a8440f58): hunt 60 · automaton 40 · ringbuffer 20 — task 40 (runs 40 · 40 · 60). Hunt and ring-buffer match the published profile; automaton dropped from 100 to 40 median (3 runs, never reached 100). Cause not yet isolated — stated plainly rather than hidden.

Plain read: novel-algorithm implementation is reliable (100 median). Multi-threaded code is the known weak spot — for concurrency tasks, generate → run the tests → regenerate once if needed.

Speed (decode, tokens/second, per phase)

These are short-context speeds (tasks send a few hundred tokens of prompt).

Deep-context sweep — what happens when the window fills (7.7K → 131K)

Latest version — re-measured 2026-09-21 (4a8440f58, defaults, every point integrity-checked):

Plain read (latest): decode is faster than the previous version at every depth (+32% at 128K); prefill wins mid-range (+9-15% at 16-33K) and is slightly behind at 64-128K.

Previous version (7.7K → 128K)

The phase table above is short-context speed. This is the other question: how fast is it when the context is actually full? One server launch per arm; we filled the window in five steps and measured decode + prefill per point — 300 output tokens each, a fresh prompt every point (no prefix-cache reuse), run at draft depth 4 and depth 6.

One Strix Halo, nothing else serving, flash attention, q8_0 KV — measured on the previous version (ROCmFPX port build). Prefill reproduced within 1–2% across both arms. *±0.03 sampling noise.

Plain read: the decay is clean and predictable. Per-token cost grows straight-line at +1.8 ms per 1K of context (linear within ±3% across the whole range) — no cliff, nothing unexpected biting. Memory stays flat across 16× context (82.3 → 82.5 GB GTT), so nothing is spilling. Prefill declines gently but never collapses (134.8 → 64.7 t/s). Single-pass acceptance wobbles run-to-run (±0.05), so don't over-read any one acceptance point. Even at 122K of packed context, you still get 6.5 t/s on one box.

Draft depth 4 vs 6: depth 6 is slower at short context (12.6 vs 13.7 t/s at 7.7K — deeper guesses cost more than they return), but the longer draft pays off once the window is packed: at 122K it edged ahead (7.1 vs 6.5 t/s), because fewer verify passes matter more when each pass is expensive. At the top end both arms are within single-pass sampling noise of each other — depth 4 stays the default; depth 6 is only interesting if you live at 100K+ of context.

Facts

Requirements — which llama.cpp build

Latest version (recommended): paoai-strix-engine, branch strix-main, commit 4a8440f58 — this is what the latest-version numbers below were measured on.

Previous version: the ROCmFPX route below still works and matches the previous-version numbers.

This model uses two things stock llama.cpp doesn't have:

• the glm5next architecture (GLM-5.3-Flash)
• the Q4_0_ROCMFP4_FAST quant type (type 101 — ROCm FP4, gfx1151-native)

Both live in guevae2/ROCmFPX, branch glm5next, commit 3345156 (glm5next arch + FP4 type merged; head of that branch — also buildable from kingjones30/ROCmFPX main once glm5next is upstreamed). Tested with GGML_VULKAN=ON on Strix Halo (gfx1151).

If you build from the wrong tree you'll see exactly one of these:

How to run (measured, locked)

llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-BALANCED.gguf \
  --host 0.0.0.0 --port 8080 -ngl 999 -c 65536 --parallel 1 \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  -fa on --cache-type-k q8_0 --cache-type-v q8_0 --lazy-mode off

Add --path <webui assets> if your build serves the WebUI from disk.

Known limits (honest)

• Threading code: ring-buffer implementations crash or deadlock in most chain runs (median 20). Single-threaded generation is strong. Regenerate-once guidance above.
• Long thinking: GLM-5.3-Flash reasons heavily; give it generous token budgets for complex tasks or answers get cut mid-thought
• Driver wall (Strix Halo): Vulkan single allocations near ~98GB can wedge the GPU (needs reboot) — this model at 71.8 GiB is safely under, but don't merge shards into one file

Serving features explained (plain words)

Decode speed drops as context fills — now measured in detail: 13.7 t/s at 7.7K → 6.5 t/s at 122K, straight-line cost growth with no cliff (full table + chart above). Stable over time. Full plain-words guide: serving-glossary.md in our repo docs.

Update log

Fleet recipes

Serving configs, ring self-heal infra, and our benchmark battery: paoai-strix-recipes (GitHub).

Credits & license

• Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
• Unsloth — BF16 source + imatrix
• kingjones30 / charlie12345 — ROCmFPX format & recipes
• ggml-org — llama.cpp
• PaoAI — glm5next port, hybrid recipe, fleet benchmarking, chain test

MIT, same as the base model. Not affiliated with Z.AI.