Mistral-Small-4-119B-ROCmFPX-Q8_0-GGUF

⛔ THIS BUILD DOES NOT FIT ON A 128 GB STRIX HALO

llama.cpp reports 113.03 GiB addressable on a Ryzen AI MAX+ 395. These 8-bit builds are 114.38 GiB and 116.15 GiB. Attempting -ngl 999 hard-wedges the machine — we did it twice: a KFD SVM D-state livelock (svm_range_cpu_invalidate_pagetables) that survives a GPU reset and needs a power cycle. -fit off does not save you; it only stops llama.cpp from shrinking the model, so it allocates until the driver dies.

On a single 128 GB Strix Halo, use the 4-bit build (63.07 GiB, 37.86 tok/s). These 8-bit builds are for machines with more memory, or for CPU / partial-offload inference.

Mistral-Small-4-119B-A6.5B — ROCmFPX 8-bit GGUF

An 8-bit ROCmFPX quantization built from BF16 (222 GiB) — a lossless source, not a requantization of a lower-bit build. 119B total / 6.5B active MoE.

Built with --output-tensor-type q8_0 --token-embedding-type q8_0 --tensor-type shexp=q8_0. The shexp override matched 108 shared-expert tensors (confirmed in the dry-run receipt — a --tensor-type pattern that matches nothing is a silent no-op, so we check the count).

⛔ Requires a llama.cpp with the ROCmFPX quant types

Q8_0_ROCMFPX (111) / Q8_0_ROCMFPX_AGENT (115) exist only in charlie12345/ROCmFPX. Stock llama.cpp reports invalid ggml type 103. Ignore the auto-generated "Use this model" commands above.

---

All quant variants

Repos: 4-bit · 8-bit AGENT · 8-bit plain

Verified

• Structural: GGUF v3, 579 tensors, 59 KV pairs — matches the source topology.
• Correctness (CPU-only, -ngl 0): 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo. Loaded in 256 s from disk.

What was NOT measured

• ⛔ No GPU decode benchmark. The build does not fit in addressable GPU memory on our hardware, so we have no tok/s figure for it and do not quote one.
• No perplexity run, no quality A/B against the source, no long-context or tool-calling tests.

Base model licence inherited; credit for the model goes to its authors.