Ornith-1.5-9B-Abliterated-ROCmFP4-GGUF

Ornith-1.5-9B-Abliterated — ROCmFP4 for AMD Strix Halo (gfx1151)

The model here is not our work. The abliteration, conversion and original validation are PocketAI Model Lab's, published as PocketAiHub/Ornith-1.5-9B-Abliterated-GGUF; the base model is ornith-ai's. This repository adds only the ROCmFP4/ROCmFPX quantisation ladder, the dual-backend build, and the measurements. Go star their repo.

Six ROCmFP4 / ROCmFPX quantisations of Ornith-1.5-9B-Abliterated, built and measured on a Ryzen AI Max+ 395 (Radeon 8060S). Every file runs on BOTH the HIP (ROCm) and Vulkan backends from a single binary — the backend is a runtime -dev flag, not a rebuild. The F16 vision projector works with every tier.

Lead result — this model is the opposite of its 35B sibling. On the 35B (MoE) the backend was worth ~10× the tier. Here, on the 9B (dense), the backends are within ~1 % on the 4-bit tiers and the interesting spread is between tiers instead. Do not carry a backend ranking across architectures.

Which file should I use?

⛔ Q6 is strictly dominated on this model — don't use it

Q6_0_ROCMFPX_AGENT is worse on every axis than the 4-bit tiers: slower (24.4 vs ~35.5 tok/s), larger (7.88 vs ~4.9 GiB), and worse perplexity (9.5850 — the worst of all six). It is also the one tier where Vulkan loses badly to ROCm (21.1 vs 24.4, −13.6 %). There is no workload on this model where Q6 is the right answer. It is published only for completeness of the ladder.

This is not a general claim about Q6 — it is what this model measured.

---

Quick start — the backend is one flag

# ROCm — the conservative default (zero truncated generations in 72)
llama-server -m Ornith-1.5-9B-Abliterated-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-Ornith-1.5-9B-Abliterated-F16.gguf \
  -dev ROCm0 -fa on -ngl 999 -fit off -np 1 \
  -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja \
  --repeat-penalty 1.05 --repeat-last-n 256 \
  --host 0.0.0.0 --port 8080

# Vulkan — SAME FILE, only -dev changes. Within ~1 % on this model.
#   -dev Vulkan0

⚠️ This model reasons before answering — give it enough max_tokens

It emits a reasoning block first, then the answer. If your token budget runs out during reasoning you get an empty content field — a valid HTTP 200 with nothing in it, which reads like a broken model.

Measured on the six tiers with a single "write a function" prompt (reasoning chars → total completion tokens):

Every tier needed more than 300 tokens for that one short request, and STRIX_LEAN needed 708. A max_tokens: 300 cap returns empty content from this model — we hit exactly that in our own first test run. Budget ≥ 1024, and read reasoning_content separately if your client exposes it.

--repeat-penalty 1.05 --repeat-last-n 256 is recommended, not decorative: with all repetition control off, this family can fall into a repeating-candidate loop on long agentic turns.

⛔ You need a ROCmFPX build with Vulkan enabled — stock llama.cpp will NOT load these files

ROCmFP4/ROCmFPX use ggml tensor types 100–119. Upstream llama.cpp's type table stops at 43, so a stock build rejects these files outright, on any backend. Build one with both backends — this is the exact configuration behind every number on this card:

cmake -S . -B build-hipvk -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGGML_NATIVE=ON \
  -DVulkan_GLSLC_EXECUTABLE=/usr/bin/glslc \
  -DAMDGPU_TARGETS=gfx1151
cmake --build build-hipvk -j

# verify BOTH backends are present before trusting any of this:
build-hipvk/bin/llama-server --list-devices   # must list ROCm0 AND Vulkan0

Traps: glslc is not always auto-detected (pass -DVulkan_GLSLC_EXECUTABLE); a stale CMakeCache poisons a generator switch (rm -rf the build dir).

---

Measured performance — all six tiers, both backends

Decode tok/s, median of 2 scored reps (warm-up discarded), n_predict = 400, -c 32768, single seat. Medians are computed over full-length generations only; any truncated generation is excluded and noted.

‡ Q8/Vulkan0 code had 1 truncated generation of 2 scored, so that single figure is a median over one surviving generation — treat it as indicative, not measured. Every other cell in this table is a median over 2 scored generations with zero truncations.

† COHERENT/Vulkan0 is reported from a deeper 48-generation re-run rather than the 2-rep protocol, because 3 of its 4 workloads truncated under Vulkan and the 2-rep medians were unusable — one workload had no valid sample at all. Medians are over full-length generations only (10, 12, 8 and 10 of 12 respectively). See the Vulkan section below.

12 of 12 arms loaded. Zero load failures on either backend.

The three findings that matter

1. Backend barely matters here — and that is the opposite of the 35B sibling. On the 4-bit tiers the two backends are within ~1 %. On the 35B (MoE) Vulkan was worth +14 %. Same family, same recipe, same box: the backend ranking did not transfer across architectures. Measure it on your model.
2. A 4-bit tier beats full Q8 on perplexity. STRIX_LEAN (4.89 GiB, 9.4313) is better than Q8_0_ROCMFPX (8.64 GiB, 9.4559) while running ~1.6× faster. The 4-bit ROCmFP4 tiers are not a quality compromise on this model.
3. Q6 loses on every axis — slower and larger than 4-bit, worse PPL than everything. See above.

---

Perplexity — measured

wikitext-2-raw, 100 chunks, -c 512, ROCm0, -ngl 999 -fa on. Lower is better.

The spread across the top five is 0.04 PPL — smaller than the ± on any single measurement. Treat those five as quality-equivalent and choose on size and speed.

---

⚠️ Vulkan silently truncated generation on this model — use ROCm

This is the most important thing on this page. Under Vulkan this model intermittently emits 1 token with empty content and stops. Under ROCm it never did once, in 120 generations.

The deep check — 48 generations per cell

Same recipe, same box, same binary, same driver — and only this model on Vulkan fails. The failure lives at the (this model × Vulkan) intersection, not in the format or the backend alone.

Per workload, on COHERENT/Vulkan0 (12 generations each):

Throughput on the surviving generations is normal — this does not look like a slow or overloaded GPU. It either produces the full answer at full speed, or produces nothing.

Across the full six-tier ladder — 12 generations per arm

Tier-variable, not tier-exclusive. It is worst on COHERENT, but it also hit Q8_0_ROCMFPX — a tier passing once does not make it immune.

It fails silently — plan for that

No error. No crash. Clean exit code. /health still returns 200. The response is simply empty (one case emitted <response> and stopped mid-tag). A benchmark that reports only median throughput cannot see this: it surfaces as an absurd tg = 1000000 tok/s (a 1-token / 0 ms decode) or as a plausible "fast" outlier inside a range. Every median on this card is therefore computed over full-length generations only, with truncations excluded and counted separately.

Recommendation for this model: run ROCm. It was clean in all 120 generations we put through it, and on the 4-bit tiers it is within ~1 % of Vulkan anyway — you give up nothing. If you do run Vulkan, count how many completions reach your n_predict rather than trusting throughput. The same GGUF serves correctly on -dev ROCm0; switching is one flag, no re-download and no rebuild.

---

Source integrity — verified before we built anything

Built from PocketAiHub/Ornith-1.5-9B-Abliterated-GGUF (BF16 GGUF), itself derived from ornith-ai/Ornith-1.5-9B pinned at revision c927ad73b7eb20f00aafcaa0a11a9d58ed5487bc.

The abliteration manifest was checked against the base before quantising — a de-fused or partially-applied checkpoint produces a model that quantises fine and then fails behaviourally, which is expensive to discover late.

---

Reproduction block

A number without its binary is a rumour. Everything on this card came from:

---

Verification — every tier, measured

12 of 12 backend load arms passed. 0 load failures. Tool calling is 18/18 correct structured calls across all six tiers. Every tier returns real, runnable code when given an adequate token budget.

The coherence probe was run at max_tokens: 1024 with reasoning_content captured separately. An earlier run of this same probe at max_tokens: 300 reported STRIX_LEAN as producing empty output — that was a probe defect, not a model defect: the server had generated a full 300 tokens, all of them reasoning. See the max_tokens note above.

Vision: the F16 projector loads against every tier. Image-grounded accuracy is not scored here — see Not yet measured.

---

Not yet measured

• Image-grounded vision accuracy (the projector loads and passes a smoke test; output is not scored against ground truth).
• Context beyond 32768.
• Speculative decoding / MTP — the source ships no MTP head.
• Full-corpus perplexity (the figures above are a 100-chunk run).
• Whether the Vulkan truncation reproduces on other gfx1151 hosts or driver versions. We saw it on one box.

---

License and attribution

Released under MIT, matching the source derivative.

• Base model: ornith-ai/Ornith-1.5-9B by the Ornith team, pinned at c927ad73b7eb20f00aafcaa0a11a9d58ed5487bc.
• Abliterated derivative and original validation: PocketAI Model Lab, published as PocketAiHub/Ornith-1.5-9B-Abliterated-GGUF.
• This repository contributes only the ROCmFP4/ROCmFPX quantisation ladder, the dual-backend build, and the measurements above.

Responsible use

This is an experimental derivative in which learned refusal behaviour has been reduced. It is published for research and legitimate local use. The edit reduces refusal broadly rather than judging whether a request is legitimate — deployers should evaluate it in their own context and apply their own safeguards. Abliteration is not truthfulness training, not a capability improvement, and not a guarantee of compliance.

Acknowledgements

The Ornith team for the base model; PocketAI Model Lab for the abliteration and its machine-readable provenance, without which the source-integrity check above would not have been possible; and the ROCmFPX project for the FP4 tensor types that make these tiers exist at all.