Granite-4.1-3B-ROCmFP4-GGUF

⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

📦 1.985 GB — 5.5% smaller than Q4_K_M, decode within noise.

Granite-4.1-3B — ROCmFP4 (tier 102 COHERENT) GGUF

A 4-bit ROCmFP4 quantization for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo), quantized from BF16 GGUF (6.81 GB) — a lossless source, not a requantization of a lower-bit build.

⛔ Requires a llama.cpp with the ROCmFP4 quant types

Q4_0_ROCMFP4_COHERENT (ftype 102) exists only in charlie12345/ROCmFPX, not upstream llama.cpp. Ignore the auto-generated "Use this model" commands above.

---

Measured

Ryzen AI MAX+ 395 (gfx1151, 128 GB unified, ROCm 7.2.4). Median of 3+, warm-up discarded, otherwise-idle box. Correctness at the model's official sampling.

⚠️ Decode ranges OVERLAP — we make no speed claim. The medians differ but the intervals cross, so this is within noise. The size saving is the real result.

Correctness: 17×23 ⇒ ✅ 391 · capital of Japan ⇒ ✅ Tokyo · days in 2024 ⇒ ✅ 366

Per-tensor types (audited in the finished file)

280× Q4_0_ROCMFP4 (type 100) · token_embd Q6_K · 81 norms F32

granite is a dense architecture here — no SSM/MoE state to protect.

---

What was NOT measured

• No perplexity run, and no quality A/B against the baseline or the source. The checks above are memorized-fact prompts — necessary but not sufficient; a damaged model can pass them.
• No long-context testing.
• No tool-calling evaluation.

Base model licence inherited; credit for the model goes to its authors.