Ornith-1.0-35B-MTP-ROCmFP4-GGUF

Ornith-1.0-35B-MTP — ROCmFP4 (Strix Halo)

⚠️ Requires the ROCmFPX fork of llama.cpp — this will NOT load on mainline llama.cpp. ROCmFP4 is an experimental AMD FP4 quant format (ggml tensor types 100–107) that only exists in that fork. Targets AMD Strix Halo / Radeon 8060S (gfx1151). For standard GGUF that runs anywhere, use the base repo: → singulared/Ornith-1.0-35B-MTP-GGUF (Q8_0 / Q4_K_M, mainline llama.cpp).

Ornith-1.0-35B (DeepReinforce, a qwen35moe agentic coder) with an embedded MTP (nextn) head grafted in, quantized to ROCmFP4-COHERENT for fast self-speculative decoding on Strix Halo.

What it is / provenance

• Body: Q4_0_ROCMFP4_COHERENT (4.70 bpw — ROCmFP4 experts + Q6_K token embeddings), quantized directly from the BF16 original (deepreinforce-ai/Ornith-1.0-35B) — a single clean quantization pass, no intermediate Q8 (avoids double-quant).
• MTP head: blk.40.nextn.* grafted raw from Qwen/Qwen3.6-35B-A3B's native head (same qwen35moe arch), after quantization — so eh_proj stays at its original Q8_0. The head's eh_proj precision is the throughput lever, and grafting it raw keeps it high. Ornith is a barely-shifted fine-tune of Qwen3.6-35B-A3B, so the head transfers cleanly (~88% draft acceptance).

Performance (Radeon 8060S / gfx1151, ROCmFPX fork · 19.2 GiB)

Recommended: Vulkan (-dev Vulkan0) — best decode, which dominates a coder's token cost:

• decode: 86.7 t/s (MTP n4, -ub 2048) · 72 t/s without MTP · draft acceptance 0.88
• prefill: ~1050 t/s (pp4096, -ub 2048)

Backend / ROCm-version comparison (-ub 2048)

• Vulkan wins decode decisively (86.7 vs 70.5) → the serving default (decode dominates a coder's token cost).
• Compiling the fork against ROCm 7.15 gives +32% ROCm prefill (876→1155, beating Vulkan's) — but it's the slowest decode, so it only helps prefill-bound work. The gain is compile-time (running a 7.2-built binary on 7.15 libs does nothing). ⚠ ROCm 7.15 is the current TheRock nightly, not a stable release — specifically therock-dist-linux-gfx1151-7.15.0a20260721 (build 2026-07-21), built against the nightly tarball, not a packaged ROCm. Newer nightlies may differ.
• Numbers are gfx1151-specific; other Strix Halo parts may differ.

Usage (ROCmFPX fork)

llama-server -m ornith-1.0-35b-MTP-ROCmFP4-COHERENT.gguf \
  -dev Vulkan0 -fa on -ngl 99 -c 131072 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.6 --alias ornith

The MTP head is embedded — no separate draft model. --spec-draft-n-max 4 is the MoE sweet spot (deeper drafts get rejected). Build the fork per its Strix Halo quickstart.

Licensing & attribution

Derivative of two permissively-licensed models; both credited, their licenses apply to their parts:

• Ornith-1.0-35B — © DeepReinforce — MIT — the base weights.
• Qwen3.6-35B-A3B — © Alibaba Cloud / Qwen — Apache-2.0 — the grafted MTP head.

Quantization: ROCmFP4-COHERENT (ROCmFPX fork). Not affiliated with or endorsed by DeepReinforce, Qwen, or the ROCmFPX author.