GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP-GGUF

GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP-GGUF

The pure-FP4 pool quant of GLM-5.3-Flash for AMD Strix Halo fleets. The whole model on the chip's native 4-bit format (159.2 GiB), designed to be split across two Strix Halo boxes over a direct link when one box isn't enough. The MTP draft head is preserved in the file — but note: on a 2-box pool, speculative decoding is OFF (pool round-trips eat the draft's gains; measured in our fleet).

Brand: PaoAI · community quant, not affiliated with Z.AI.

---

Who is this for?

You have two or more Strix Halo boxes connected directly (USB4/10GbE) and want the full GLM-5.3-Flash in native FP4 split across them. If you have ONE box, use our STRIX-BALANCED instead — a pool of this model loses to a solo smaller quant on both speed and simplicity.

The recipe

Measured on real hardware

Build source (required)

Both binaries need the glm5next arch and the FP4 type. Both live in guevae2/ROCmFPX, branch glm5next, commit 3345156 (glm5next arch + FP4 type merged; head of that branch — also buildable from kingjones30/ROCmFPX main once glm5next is upstreamed). Build with GGML_VULKAN=ON on Strix Halo (gfx1151).

How to run (2-box pool, direct link)

# halo-2 (worker):
ggml-rpc-server --host 0.0.0.0 --port 50052
# BG1 (head):
llama-server -m GLM-5.3-Flash-PaoAI-ROCmFP4-STRIX-MTP.gguf \
  --host 0.0.0.0 --port 8081 -ngl 99 -c 32768 --parallel 1 \
  --rpc <worker-ip>:50052 -fa on

⚠️ Pool endpoint rule (learned the hard way): use the DIRECT box-to-box link addresses, never routes through a third machine — RPC traffic via a middle box deadlocks after load (head stuck in send, worker in recv).

Honest position

This is the largest and least convenient of our three models: it needs two boxes, delivers ~5.4 t/s, and can't run solo. We publish it because it completes the recipe set for fleet owners — and because the measurement behind it (pools reward lean bytes-per-token over small file size) is one of the core findings of our work.

Credits & license

• Z.AI — GLM-5.3-Flash base model (MIT license; included in repo)
• Unsloth — BF16 source + imatrix
• kingjones30 / charlie12345 — ROCmFPX format & recipes
• ggml-org — llama.cpp
• PaoAI — glm5next port, FP4 recipe, fleet benchmarking

MIT, same as the base model. Not affiliated with Z.AI.