Qwen3.6-27B-Q8_0-ROCMFPX-GGUF

Qwen3.6-27B — Q8_0_ROCMFPX GGUF (experimental, AMD gfx1201)

An experimental Q8 quantization of Qwen/Qwen3.6-27B, tuned and validated for the AMD Radeon AI PRO R9700 (gfx1201).

[!IMPORTANT] This file does not run on upstream llama.cpp, Ollama, LM Studio, or vLLM. It uses the custom Q8_0_ROCMFPX tensor type and requires the pinned ROCmFPX fork build described below. Unsupported runtimes should reject the file; if a tool appears to load it anyway, do not trust the output. Hugging Face's GGUF metadata viewer may also mislabel the custom tensor type or fail to parse the file.

Should you use this?

Use this model if all of the following are true:

• you run a Radeon AI PRO R9700 (gfx1201) under ROCm;
• you are willing to build the pinned fork from source;
• you want a Q8 that is 2.94% smaller than upstream Q8_0 and performs equal observed model work about 6% faster on this hardware.

Otherwise, use an ordinary Q8_0 GGUF of Qwen3.6-27B. It has broad support in the standard GGUF runtime ecosystem, and its quality is equivalent within the tolerances measured here.

TL;DR — measured against upstream Q8_0

The supported conclusion is deliberately narrow: this format performs equal observed model work faster than upstream Q8_0 on the measured deployment. The campaign did not confirm that complete live-agent jobs finish at least 3% faster, so no such claim is made. All thresholds were fixed before the runs; failed gates are reported, not reinterpreted.

The public claim registry records the scope, result class, and sealed aggregate-evidence hash behind each release claim.

What Q8_0_ROCMFPX actually is

Each block stores 32 signed eight-bit codes and one finite UE4M3 scale byte: 33 bytes per 32 weights, or 8.25 bits per weight. The HIP path decodes the scale and performs integer MMVQ/MMQ dot products with float accumulation. This is not native FP8 or FP4 matrix arithmetic.

GGUF block
  32 signed int8 codes + UE4M3 scale
        |
        v
activation quantization to Q8_1
        |
        +--> decode:  gfx1201 MMVQ, VDR8 + measured wave policy
        |
        `--> prefill: integer MMQ path
        |
        v
float accumulation / model output

The experimental format and execution path come from the upstream ROCmFPX project. This repository's contribution is the gfx1201 decode tuning (VDR8 vector-dot width and a measured wave policy) and the validation below; other GPU targets retain ROCmFPX defaults and are unmeasured.

The quantization is uniform Q8 with no importance matrix and no per-tensor routing.

Required runtime

Build the pinned fork branch:

git clone https://github.com/1337hero/ROCmFPX.git
cd ROCmFPX
git checkout 45bcff509c4b1cff137e2cc1ea84671c61ceddea
env JOBS=16 scripts/build-r9700.sh

The wrapper is the simplest supported build. The sealed performance runners used this HIP-only configuration:

cmake -S . -B build-r9700 -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON \
  -DGGML_HIP_FORCE_MMQ=ON \
  -DGGML_VULKAN=OFF \
  -DGGML_CUDA=OFF \
  -DCMAKE_HIP_ARCHITECTURES=gfx1201 \
  -DGPU_TARGETS=gfx1201 \
  -DCMAKE_HIP_FLAGS= \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_BUILD_WEBUI=OFF \
  -DLLAMA_USE_PREBUILT_WEBUI=OFF \
  -DLLAMA_BUILD_TESTS=ON \
  -DGGML_BUILD_TESTS=OFF
cmake --build build-r9700 -j 16 --target \
  llama-server llama-bench llama-quantize

The validated implementation is ROCmFPX base commit 6bf20cd688ba0af882d1f68ba50b292edf646ab4 plus commits eb38c6f67701ff9c74e8597f573eedf9ccecf774 and 45bcff509c4b1cff137e2cc1ea84671c61ceddea.

Binaries validated by the lab:

Example deployment

This shape matches the validated two-card, 262,144-token envelope. Device names depend on the host; check --list-devices first.

./build-r9700/bin/llama-server \
  -m Qwen3.6-27B-Q8_0_ROCMFPX.gguf \
  --no-mmap \
  -c 262144 \
  -b 2048 \
  -ub 512 \
  -t 16 \
  -ngl 99 \
  -sm layer \
  -ts 1,1 \
  -dev ROCm0,ROCm2 \
  --fit off \
  -ctk f16 \
  -ctv f16 \
  -fa on \
  -np 1

Both this model and the upstream Q8_0 control allocated one 262,144-token slot across two R9700s with F16 K/V cache and generated bounded valid output; this model used 804,319,232 fewer resident bytes in the captured snapshots. That test proved allocation and bounded generation, not a timed full-context prompt.

Measurement conditions

All performance numbers were measured on:

• Qwen3.6-27B, this GGUF vs an upstream Q8_0 control of the same conversion, same host, same workload;
• two non-display Radeon AI PRO R9700s at a pre-existing 250 W, −75 mV tune — not stock 300 W;
• ROCm 7.2.4 on Arch Linux, outside the official Radeon Linux support matrix;
• pinned source revisions, forced model residency, reversed candidate order, and predeclared promotion thresholds.

The equal-work result comes from 18 common-seed matched agent-workload pairs with ABBA/BAAB counterbalancing, warmup before every measured run, full request accounting (1,899 logical requests, zero retries), and exact paired Wilcoxon signed-rank tests.

Artifact

Quantization command (the qualifying run wrote to a temporary filename and then promoted the verified hash without overwriting an existing artifact):

./build-r9700/bin/llama-quantize \
  Qwen3.6-27B-BF16.gguf \
  Qwen3.6-27B-Q8_0_ROCMFPX.gguf \
  Q8_0_ROCMFPX \
  16

Verify after download:

sha256sum -c SHA256SUMS

Limitations

• Requires an experimental fork; upstream llama.cpp compatibility awaits a separate code contribution.
• Validated on exactly one model conversion, one Arch Linux host, ROCm 7.2.4, and gfx1201 GPUs at a tuned power profile. Other GPUs, hosts, and stock power are unmeasured.
• Aggregate quality stays within the declared tolerances, but this model and upstream Q8_0 do not produce identical per-task outcomes.
• The raw private lab archive (prompts, outputs, paths, environment) is not published; it awaits a separate sanitization review.

License and attribution

• Base model: Qwen3.6-27B, Copyright 2026 Alibaba Cloud, Apache-2.0. This repository redistributes a converted and quantized derivative under the same license. See LICENSE and NOTICE.
• Format and execution path: the experimental Q8_0_ROCMFPX representation and HIP kernels are the work of the ROCmFPX project, which builds on llama.cpp.
• This repository: the gfx1201 decode tuning, quantized artifact, and validation evidence.

Qwen and related marks belong to their owners. This community quantization is not affiliated with or endorsed by Qwen, Alibaba Cloud, AMD, ROCmFPX, or llama.cpp.