Xing4.0-29B-A4B-ROCmFP4-FAST-imatrix-GGUF

Xing4.0-29B-A4B — ROCmFP4 FAST + imatrix (Strix Halo / gfx1151)

The single importance-matrix build of XingChen-AGI/Xing4.0-29B-A4B, on the FAST tier — the fastest-decoding and smallest of the three, which is also the tier an imatrix has the most room to help.

Standard (non-imatrix) tiers: Xing4.0-29B-A4B-ROCmFP4-GGUF.

⛔ Needs the ROCmFPX fork — stock llama.cpp cannot load this

ROCmFP4 tensor types are ≥ 100 (past stock GGML_TYPE_COUNT), and xing4 is a new architecture (mHC hyper-connections + MLA with n_embd_head_k=192 / n_embd_head_v=128). Build from the xing4-port branch of our fork (pinned at cbc2cd5) — main does not carry xing4:

git clone -b xing4-port https://github.com/kingjones30/ROCmFPX.git

Files

The .imatrix is shipped so you can reproduce or re-target the quantization yourself.

What the imatrix actually bought — read this before assuming

−0.8% is a modest gain, and it does not make FAST the best-quality file in the family. COHERENT-noimatrix still measures better. Choose this build for speed and size; choose COHERENT if quality is the priority.

For context, the same pipeline gained −5.9% on a different MoE. The likely reason it is small here: calibration was English wikitext against a Chinese-first model. A Chinese or mixed calibration set would probably do better, and the shipped .imatrix plus the recipe below let you try exactly that.

The imatrix was genuinely applied, not silently ignored — llama-quantize reported "loaded 552 importance matrix entries … computed on 598 chunks".

Usage

llama-server -m Xing4.0-29B-A4B-Q4_0-ROCmFP4-FAST-imatrix.gguf \
  -c 32768 -ngl 99 -fit off --jinja

⚠️ Reasoning model — budget the tokens

Thinking is on by default. A two-sentence answer took 569 completion tokens; max_tokens: 200 returns empty content. Use ≥ 600. reasoning_content is separated correctly by llama-server. The trace is frequently Chinese even for English answers — expected.

Tool calling

Verified on this exact file:

finish_reason: tool_calls
get_weather({"city":"Tokyo","unit":"celsius"})

Verification

The port was validated against upstream transformers on identical tokens:

⚠️ llama-perplexity scores only the second half of each window; a naive script scoring from position 1 gives 11.91 for the same model and data. Match the window before comparing.

Not tested

• Contexts beyond 32K (262144 cannot be allocated on 128 GiB hardware — see MTP), and the Vulkan backend.

MTP: measured, and it is the biggest lever on this model

Layer 40 is the model's MTP block and its weights are in these files (block_count = 41, nextn_predict_layers = 1). Measured 2026-09-19 on the STRIX_LEAN file, same flags, 589-token prompt generating 160 tokens, 32K context, quiet box, ROCm0:

Use --spec-draft-n-max 1. n-max 2 accepts slightly longer drafts (1.83 vs 1.65) and is still much slower, because the extra draft work costs more than it returns. Prefill pays 14.5% for the gain, so MTP favours generation-heavy traffic over long-prompt batch work.

⛔ MTP costs you half your context

The MTP draft context allocates its own full-size KV cache. Two consequences measured on a 128 GiB Strix Halo:

• The declared 262144 context is not reachable on this hardware at all: the KV allocation asks for 209,920 MiB (205 GiB) and fails. That is ~0.80 MiB per token, uncompressed.
• With MTP enabled, 131072 still fails (failed to create MTP context) and 32768 is what fits.

⚠️ -ctk q8_0 -ctv q8_0 is silently ignored on this arch. At 131072 with both flags set the allocation was 104,960 MiB — exactly half the 262144 figure, i.e. pure context scaling with no quantisation benefit. Do not plan context around quantised KV here.

Reproduction

box     AMD Ryzen AI Max+ 395 "Strix Halo", gfx1151, ROCm 7.2.4, 123 GiB unified
build   github.com/kingjones30/ROCmFPX, arch xing4
source  XingChen-AGI/Xing4.0-29B-A4B -> 936-tensor BF16 GGUF (62.4 GB)

imatrix llama-imatrix -m <BF16>.gguf -f wiki.test.raw -c 512 -ub 64 -ngl 99 -fit off
        -> 552 entries over 598 chunks
quant   llama-quantize --imatrix xing4.imatrix --output-tensor-type q4_K <BF16> <out> 103 32

ppl     llama-perplexity -m <f> -f wiki.test.raw -c 512 --chunks 32 -ub 64 -ngl 99 -fit off
speed   llama-bench -m <f> -p 512 -n 128 -ngl 99 -r 3     (quiet box)

⛔ -ub 64 in the imatrix command is load-bearing, not a preference. On gfx1151 the ROCm mul_mat_id sorted fallback computes wrong values for unquantized MoE, and the imatrix pass runs on BF16. Built at the default -ub 512, this model's imatrix produced PPL 318,130; at -ub 64 it produced a valid one. Reproducer: test-backend-ops -o MUL_MAT_ID → 1311/1320, all 9 failures f16/bf16, all 920 quantized cases pass (so the shipped file here is unaffected).

Credits

Base model © China Telecom Artificial Intelligence Technology Co., Ltd. (XingChen-AGI · GitHub), Apache-2.0. Quantized and arch-ported by kingjones777; this GGUF inherits Apache-2.0.