Qwen3.8-27B-abliterated-ROCmFPX

Qwen3.8-27B-abliterated-ROCmFPX

ROCmFPX quantized GGUF (7 tiers + vision mmproj) of huihui-ai/Huihui-Qwen3.8-27B-abliterated — abliterated BF16 weights, locally requantized.

Engine: strix.cpp (tuned for gfx1151 / AI MAX 395)

This repository uses strix.cpp, a llama.cpp fork specifically tuned for the AMD Strix Halo gfx1151 (Ryzen AI MAX+ 395) UMA architecture and ROCm/HIP backend, with native support for the ROCmFPX quantization format.

• Upstream llama.cpp cannot load the ROCmFPX files in this repository.
• Upstream llama.cpp also does not carry this gfx1151-specific tuning.
• llama.cpp-rocm in the tables is the pre-rename build (build 11107); strix.cpp is the current build (build f879d83c).

⚡ Quick Start

Requires strix.cpp (this fork supports ROCmFPX; upstream llama.cpp cannot load these files — format note at the end).

# Single machine (smallest tier 13.56 GiB)
strix-server -m Qwen3.8-27B-abliterated_Q4_0_ROCMFP4_FAST.gguf -ngl 99 -fa on --port 8080

# Vision: add --mmproj mmproj-Qwen3.8-27B-abliterated_BF16.gguf
# All tiers fit on a single 128 GB machine (largest: 26.69 GiB)

🎯 Which tier? (3-second decision)

• Fastest: Q4_0_ROCMFP4_FAST
• Q4 speed with stabler quality: Q4_0_ROCMFP4_STRIX
• Quality/speed balance: Q6_0_ROCMFPX
• Instruction following / Agent: Q6_0_ROCMFPX_AGENT / Q8_0_ROCMFPX_AGENT
• Highest quality: Q8_0_ROCMFPX

Model Overview

• Base model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
• Architecture: dense 27B — all parameters active; tg is inherently lower than MoE models
• Quantization: 7 tiers — Q4_0_ROCMFP4 (FAST / FAST_COHERENT / STRIX), Q6_0_ROCMFPX (base / AGENT), Q8_0_ROCMFPX (base / AGENT)
• MTP: built-in MTP head — enable with strix-server --spec-type draft-mtp (off by default)
• Vision: bundled mmproj (CLIP + qwen3vl_merger vision tower)
• Context: up to 256K

Files

SHA-256 checksums for all files: see sha256.txt.

Quantization Tiers

Benchmarks (llama-bench)

Comparison: llama.cpp-rocm vs strix.cpp

Measured on AMD Strix Halo (gfx1151) with the same prompt and generation lengths:

llama-bench -p 256,2048 -n 256 -r 5 -fa 1 -ngl 999 -ub 512

llama.cpp-rocm is build 11107; strix.cpp is build f879d83c. llama.cpp-rocm numbers are the mean of the last 4 of 5 shots; strix.cpp numbers are the mean of 5 shots (first shot included).

pp256 (t/s)

pp2048 (t/s)

tg256 (t/s)

MTP (strix.cpp)

Usage

① Single machine (all tiers)

strix-server -m Qwen3.8-27B-abliterated_Q4_0_ROCMFP4_FAST.gguf -ngl 99 -fa on --port 8080

# Vision: add --mmproj mmproj-Qwen3.8-27B-abliterated_BF16.gguf

• OpenAI-compatible API: http://127.0.0.1:8080/v1
• Vision is enabled with --mmproj
• MTP: strix-server --spec-type draft-mtp (off by default)
• Context: up to 256K

② MTP

strix-server -m Qwen3.8-27B-abliterated_Q4_0_ROCMFP4_FAST.gguf \
  --spec-type draft-mtp --spec-draft-n-max 6 -ngl 99 -fa on --port 8080

MTP is useful for latency-sensitive single-stream generation on some models; benchmark before enabling it.

Tech Notes (advanced)

• strix.cpp is tuned for the gfx1151 UMA architecture and ROCm/HIP backend; -ngl 999 places all layers on the GPU.
• MTP uses the model's built-in nextn head, not an external draft model.

Format Note (ROCmFPX)

ROCmFPX is a proprietary quantization format (GGML types 100–107, a strix.cpp extension) — upstream llama.cpp cannot load these files. Load them with strix.cpp, a llama.cpp fork that supports all upstream GGUF formats plus ROCmFPX natively.

License & Disclaimer

• Base model: Qwen/Qwen3.8-27B — Apache-2.0
• Abliterated version: huihui-ai/Huihui-Qwen3.8-27B-abliterated — Apache-2.0
• This quantization: Apache-2.0 (ArtomYuan)

This is an abliterated (uncensored) version and may produce outputs that do not conform to safety policies. Use at your own risk. No warranty is provided.