AEON-Ultimate-ROCmFP4-Strix-Halo-GGUF

AEON Ultimate ROCmFP4/ROCmFP6 GGUF for AMD Strix Halo

This repo contains AMD Strix Halo GGUF quantizations of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16, including the bundled Qwen3.6 MTP tensors for llama.cpp speculative decoding.

These files require custom Strix Halo runtimes. They are not for stock llama.cpp and not for vLLM.

• ROCmFP4 runtime: charlie12345/rocmfp4-llama, branch mtp-rocmfp4-strix.
• ROCmFP6 runtime: charlie12345/ROCmFPX, tested at commit 11d76c2.

Files

All variants were quantized from the BF16 source GGUF with the MTP extension bundled. Do not requantize from the FP4 files.

Which File Should I Use?

For lowest latency on one Strix Halo, start with AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf. It had the best measured MTP decode speed in my coding-agent tests.

For a higher-quality FP6 experiment, start with AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf. It preserves more tensors at Q8 under ROCmFPX's agent routing and is the most plausible coding-harness candidate among the FP6 files. It is slower than default FP6 in a short decode smoke test.

The FP6 variants are uploaded as split GGUFs with shards under 4 GB to avoid large multipart-upload issues. Put all shards for a variant in the same directory and load the 00001-of-... file; llama.cpp will find the remaining shards.

For parallel subagent pools, test no speculative decoding as well as MTP. In my FP4 tests, MTP helped single-stream latency but did not scale well across four parallel slots.

Build the ROCmFP4 Runtime

git clone https://github.com/charlie12345/rocmfp4-llama.git
cd rocmfp4-llama
git checkout mtp-rocmfp4-strix
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh

Build the ROCmFP6 Runtime

git clone https://github.com/charlie12345/ROCmFPX.git
cd ROCmFPX
git checkout 11d76c2
env JOBS=16 scripts/build-strix-rocmfp4-mtp.sh llama-cli llama-server llama-bench

The build directory is still named build-strix-rocmfp4 because ROCmFPX extends the same Strix Halo llama.cpp fork.

Common Strix Halo environment:

export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1
export LD_LIBRARY_PATH=$PWD/build-strix-rocmfp4/bin:/opt/rocm/lib:${LD_LIBRARY_PATH:-}

Recommended FP6 Coding Server

ROCmFPX includes a helper that auto-checks MTP metadata and starts llama-server with conservative draft-mtp settings:

cd ROCmFPX
MODEL=/path/to/AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf \
DEVICE=ROCm0 \
SPEC_DRAFT_DEVICE=ROCm0 \
CTX_SIZE=32768 \
PARALLEL=1 \
SPEC_DRAFT_N_MAX=4 \
PERF_PRESET=latency \
scripts/run-rocmfpx-mtp-server.sh

The helper runs with --reasoning off, --reasoning-format none, Jinja templates, full offload, flash attention, F16 KV cache, and draft-mtp enabled. That matters for OpenAI-compatible coding harnesses: with reasoning enabled, Qwen-style reasoning can be routed to reasoning_content while normal content appears empty to clients.

Equivalent core server args:

./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP6-AGENT-MTP.gguf-00001-of-00007.gguf \
  --alias aeon-fp6-agent --host 127.0.0.1 --port 8080 \
  --jinja --reasoning off --reasoning-format none \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 2048 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 --spec-draft-ngl all \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 4 --spec-draft-n-min 0 \
  --spec-draft-p-min 0.0 --spec-draft-p-split 0.10 \
  --parallel 1 --metrics --no-webui

Recommended FP4 Single-Stream Coding Server

This was the fastest tested single-stream coding-agent config:

./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf \
  --alias aeon --host 127.0.0.1 --port 8080 \
  --jinja -rea off \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 512 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 --spec-draft-ngl all \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
  --parallel 1 --metrics --no-mmap

Recommended Parallel Subagent Server

For a second Strix Halo dedicated to several lower-priority subagents, test no speculative decoding with four parallel slots:

./build-strix-rocmfp4/bin/llama-server \
  -m /path/to/AEON-Ultimate-ROCmFP4-STRIX_LEAN-MTP.gguf \
  --alias aeon --host 127.0.0.1 --port 8081 \
  --jinja -rea off \
  -c 32768 -ngl 999 -fa on -dev ROCm0 \
  -b 512 -ub 512 -t 16 -tb 32 \
  -ctk f16 -ctv f16 \
  --parallel 4 --metrics --no-mmap

Measured Strix Halo Performance

Hardware: Ryzen AI Max+ 395 / Radeon 8060S, gfx1151, 128 GB unified memory.

FP4 runtime: rocmfp4-llama branch mtp-rocmfp4-strix, commit 4795079b plus a small local converter fix. Single cold coding prompt, 256 generated tokens:

Longer-context FP4 checks with the recommended single-stream config:

Parallel/subagent FP4 serving, varied coding prompts, 256 generated tokens per request:

FP6 runtime smoke test: ROCmFPX commit 11d76c2, llama-bench -dev ROCm0 -ngl 999 -fa on -p 64 -n 64 -r 1:

These FP6 numbers are loader/throughput smoke tests, not full coding-agent MTP acceptance sweeps.

STRIX vs STRIX_LEAN

STRIX is a quality-biased ROCmFP4 preset and STRIX_LEAN is a speed/size-biased preset. They are quantizations of the same AEON Ultimate weights.

On the tested Strix Halo, STRIX did not improve speed or MTP acceptance:

Use STRIX_LEAN unless your own correctness eval shows a quality win for STRIX.

Provenance

• Source model: AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16
• Base model: Qwen/Qwen3.6-27B
• FP4 runtime fork: charlie12345/rocmfp4-llama
• FP6 runtime fork: charlie12345/ROCmFPX
• BF16-to-GGUF conversion source revision: d52a7bc77bf9c6c1bcff638bf20037e8dcb9af5b
• ROCmFPX quantization revision: 11d76c2

License

Apache-2.0, inherited from the source model and Qwen/Qwen3.6-27B.