ZAYA1-8B-ROCmFP4-GGUF

🔧 Runtime: build the ROCmFPX fork below

Stock llama.cpp will not load this file. You need both the zaya architecture and the ROCmFP4 tensor types in one tree. Upstream charlie12345/ROCmFPX has the ROCmFP4 types but not zaya. Our fork has both:

kingjones30/ROCmFPX — a fork of charlie12345/ROCmFPX, branch main.

git clone https://github.com/kingjones30/ROCmFPX.git
cd ROCmFPX
cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server llama-quantize -j$(nproc)

Verified 2026-08-27 on gfx1151: clean clone → 0 build errors → llama-server loads a zaya ROCmFP4 GGUF from this family and generates coherent text.

⚠️ STOCK llama.cpp WILL NOT LOAD THIS MODEL

The zaya architecture is not merged upstream (draft PR #23112). Ignore the auto-generated "Use this model" commands above — build the ROCmFPX fork linked just below.

📦 4.86 GiB, smaller than Q4_K_M (5.19 GiB), from a 16.52 GiB BF16 source. ⚠️ Speed is parity, not a win — see the honest benchmark below.

ZAYA1-8B — ROCmFP4 (tier 102 COHERENT) GGUF

A 4-bit ROCmFP4 quantization of Zyphra/ZAYA1-8B for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo).

---

⚠️ Read this before choosing 4-bit for ZAYA1

ZAYA1 is not weight-bandwidth-bound, so 4-bit buys size — not speed. We measured all three builds on the same machine, median of 3, warm-up discarded, on an otherwise idle box:

BF16 reads 3.4× more bytes per token and decodes at the same rate. A bandwidth-bound model would show roughly 3× separation here; ZAYA1 shows none. The bottleneck is compute — the Compressed Convolutional Attention grouped-conv path (cca_conv_grp, 10 groups per even layer).

All three ranges overlap, so we make no speed claim in either direction — this build is neither meaningfully faster nor slower than Q4_K_M. Take it for the 0.32 GiB size saving and the fact that it fits alongside more models, not for throughput.

Where the time actually goes

We localized decode cost by structurally skipping the CCA grouped convolution and re-measuring (rocprof is unavailable on this hardware):

The CCA grouped convolution is roughly 55% of decode time, and it is identical across quantizations — the conv weights are BF16 in both. Fitting t = fixed + bytes/BW to the BF16 and 4-bit points puts ~37 ms of ~42 ms per token in weight-independent fixed cost, i.e. decode is ~88% independent of the weight format.

⇒ No 4-bit format can make this model faster. A proper grouped-conv kernel could be worth up to ~2× end-to-end, but that is a llama.cpp kernel/conversion fix, not a quantization one. We prototyped a single-GEMM rewrite and did not ship it — it was not numerics-identical, and a speedup that changes output is not a speedup.

Correctness

Official-sampling checks, reading both reasoning_content and content with a 2500-token budget (ZAYA1 reasons at length — a small cap will make it look broken when it is not):

Per-tensor types (audited in the finished file, 1283 tensors)

⚠️ tie_word_embeddings is true on this model, so there is no separate output.weight and --output-tensor-type is a silent no-op. --token-embedding-type q6_K is the flag that actually lands — and on a 262,272-token vocabulary that matters a great deal.

cca_conv_grp is excluded from quantization in the patch. Those tensors drive the compressed convolutional attention and are highly sensitive; other ZAYA1 quantizers exclude them too.

---

What was NOT measured

• No perplexity run, and no quality A/B against Q4_K_M or BF16. The checks above are memorized-fact prompts — necessary but not sufficient.
• No long-context testing (the model supports 131,072).
• No tool-calling evaluation.
• No coding or reasoning benchmark.

Model

ZayaForCausalLM / zaya. 40 layers, all hybrid · hidden 2048 · vocab 262,272 · 16 experts, 1 active · ssm_d_conv = 2 · 1283 tensors · context 131,072 · tie_word_embeddings: true. ~8.4B total / ~760M active.

Base model licence: Apache-2.0 (inherited). Credit for the model itself goes to Zyphra.

All quant variants

Three builds of this model, all measured in one session on one box with one binary (Ryzen AI MAX+ 395, gfx1151, ROCm 7.2.4, ROCmFPX-2809dc5) — so these rows are directly comparable. Median of 3, warm-up discarded, otherwise-idle box.

⚠️ Decode is ~88% weight-independent on this architecture (the CCA grouped conv is ~55% of decode). All three builds land within ~10% of each other; the 4-bit is smallest and marginally fastest. No 8-bit or 4-bit format will make this model meaningfully faster.

What AGENT actually changes: it keeps far more tensors at true Q8_0 instead of the packed 8-bit type — measured in these files, 154 tensors vs 1 tensor. On models with an MTP draft head that raises draft acceptance and wins ~6%; these two models have no MTP head, and here the two 8-bit builds are within noise of each other.