Qwen3.8-27B-Heretic-ROCmFPX-STRIX

Qwen3.8-27B-Heretic-ROCmFPX-STRIX

ROCmFPX / ActiveFPX build of trohrbaugh/Qwen3.8-27B-heretic-ara, targeting AMD Strix Halo / Radeon 8060S (gfx1151) and the specialized CIRU ROCmFPX + PromptForge inference stack.

This repository contains:

• a compact ROCmFPX / ActiveFPX GGUF using the CIRU Qwen3.8 v3-style precision allocation;
• a model-specific full-W8 PromptForge FFN sidecar;
• a model-specific full-W8 PromptForge Gated DeltaNet sidecar.

The GGUF can be used on its own with a compatible ROCmFPX-capable llama.cpp runtime. The .pfs files are optional serving-time compute views for CIRU PromptForge and trade additional memory for substantially faster supported prompt-processing paths.

Model provenance

Original model

The architecture and original model weights originate from:

Qwen/Qwen3.8-27B

Qwen3.8-27B is a dense 27B model with:

• 64 language-model layers;
• hidden dimension 5120;
• FFN intermediate dimension 17,408;
• a repeating hybrid Gated DeltaNet / gated-attention layout;
• native MTP support;
• a 248,320-token padded vocabulary;
• native 262,144-token context;
• native multimodal architecture.

Heretic / ARA source

The full-precision source used for this repository is:

trohrbaugh/Qwen3.8-27B-heretic-ara

That model is a behavior-modified Qwen3.8-27B produced using the Heretic project and its Arbitrary-Rank Ablation (ARA) approach.

This repository does not perform an additional fine-tune. The ROCmFPX GGUF and PromptForge sidecars were generated from the Heretic-Ara BF16 weights.

Quantization and inference provenance

The quantization format and specialized inference implementation come from:

• CIRU ROCmFPX
• CIRU Qwen3.8 ActiveFPX + PromptForge reference release
• llama.cpp
• AMD Composable Kernel

CIRU's Qwen3.8 work adds model-specific gfx1151 paths including:

• fused FFN gate/up processing;
• accelerated SwiGLU/down projection;
• merged QKV/Z projection for Gated DeltaNet layers;
• prepacked W8 PromptForge compute views;
• small-row W8 execution;
• ROCmFPX low-bit decode paths;
• native Qwen3.8 MTP support.

About this v3-style quant

The compact GGUF in this repository was generated from the Heretic-Ara BF16 source using the per-tensor precision allocation from CIRU's Qwen3.8 v3 release as the reference.

That includes the important v3 storage decisions such as:

• Q8 output projection;
• selected LateQ6 precision islands;
• the remaining mixed ROCmFPX tensor allocation.

FFNEQ note

CIRU's official v3 model is named:

Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3-Q8-FFNEQ-LateQ6.gguf

The FFNEQ part refers to an FFN-equalization treatment in CIRU's model-production process.

This Heretic build reproduces the v3 tensor-type / precision allocation, but does not claim to reproduce CIRU's unpublished FFNEQ weight transformation unless explicitly stated otherwise.

In other words:

Heretic-Ara BF16
       |
       +--> CIRU v3-style Q8 / LateQ6 / ROCmFPX allocation
       |         |
       |         +--> compact GGUF
       |
       +--> BF16-derived PromptForge W8 views
                 |
                 +--> FFN.pfs
                 +--> GDN.pfs

PromptForge sidecars

The .pfs files are not independent models and should not be mixed with another Qwen3.8 checkpoint.

They are prepacked W8 compute views generated from this model's BF16 weights.

FFN sidecar

Contains W8 prompt-time views of:

ffn_gate
ffn_up
ffn_down

for all 64 language-model layers.

The fixed Qwen3.8 PromptForge FFN ABI uses a file size of:

17,123,004,416 bytes

GDN sidecar

Contains the merged W8:

attn_qkv + attn_gate/Z

views used by the 48 Gated DeltaNet layers.

The fixed GDN sidecar size is:

4,029,685,760 bytes

Output-K8 status

CIRU's official v3 release also includes a third sidecar:

PromptForge-Output-K8.pfs

That file provides a draft-only top-8 output shortlist used by the v3 MTP path before authoritative Q8 reranking.

This repository does not currently include a Heretic-specific Output-K8 proxy.

Do not use CIRU's Output-K8 sidecar with this model: it was generated from CIRU's output weights, not the Heretic-Ara weights.

The FFN and GDN PromptForge paths work independently of Output-K8.

---

Running with the CIRU PromptForge runtime

The optimized path is intended primarily for AMD Strix Halo / Radeon 8060S (gfx1151).

CIRU developed and validated the Qwen3.8 PromptForge work with a pinned TheRock ROCm 7.15 environment and a pinned Composable Kernel revision.

A sufficiently compatible ROCm toolchain with working gfx1151 HIP support is required.

1. Download this repository

Install the Hugging Face CLI if necessary:

python3 -m pip install -U huggingface_hub

Download the repository:

export MODEL_DIR="$HOME/models/Qwen3.8-27B-Heretic-ROCmFPX-STRIX"

mkdir -p "$MODEL_DIR"

hf download \
  tonyrishwain/Qwen3.8-27B-Heretic-ROCmFPX-STRIX \
  --local-dir "$MODEL_DIR"

Find the artifacts:

export MODEL="$(find "$MODEL_DIR" -maxdepth 1 -type f -name '*.gguf' | head -n1)"
export FFN="$(find "$MODEL_DIR" -maxdepth 1 -type f -iname '*ffn*.pfs' | head -n1)"
export GDN="$(find "$MODEL_DIR" -maxdepth 1 -type f -iname '*gdn*.pfs' | head -n1)"

printf 'MODEL=%s\nFFN=%s\nGDN=%s\n' \
  "$MODEL" "$FFN" "$GDN"

Verify the PFS sizes:

stat -c '%n %s' "$FFN" "$GDN"

Expected:

FFN: 17123004416
GDN: 4029685760

2. Install build dependencies

Fedora

sudo dnf install -y \
  @development-tools \
  cmake \
  ninja-build \
  git \
  python3 \
  libcurl-devel

You also need a ROCm/HIP installation capable of compiling and running gfx1151.

CIRU's published v3 measurements use ROCm 7.15. Distribution-provided ROCm versions may differ.

Verify your GPU:

rocminfo | grep -m1 gfx1151

You should see gfx1151.

3. Download CIRU's v3 runtime patch

The v3 runtime is built from CIRU's public v2.3 tag plus the v3 Output-K8 patch.

Even though this model does not currently use Output-K8, using the current patched runtime keeps the execution stack aligned with CIRU v3.

export CIRU_PATCH="$HOME/ciru-qwen38-v3"

mkdir -p "$CIRU_PATCH"

hf download \
  jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge \
  runtime/qwen38-v3-output-k8-runtime.patch \
  --local-dir "$CIRU_PATCH"

4. Clone ROCmFPX

mkdir -p "$HOME/src"
cd "$HOME/src"

git clone https://github.com/ciru-ai/ROCmFPX.git
cd ROCmFPX

git checkout qwen3.8-activefpx-promptforge-v2.3

git am \
  "$CIRU_PATCH/runtime/qwen38-v3-output-k8-runtime.patch"

5. Clone the pinned Composable Kernel

cd "$HOME/src"

git clone https://github.com/ROCm/composable_kernel.git

git -C composable_kernel checkout \
  fdf4bb7fcc984811cef48ce817d89aac064b984a

Set:

export ROCMFPX="$HOME/src/ROCmFPX"
export CK="$HOME/src/composable_kernel"

6. Configure ROCm

If your ROCm installation is under /opt/rocm:

export ROCM_PATH=/opt/rocm
export PATH="$ROCM_PATH/bin:$ROCM_PATH/llvm/bin:$PATH"
export LD_LIBRARY_PATH="$ROCM_PATH/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

Check:

"$ROCM_PATH/bin/rocminfo" | grep -m1 gfx1151

On Fedora or a custom TheRock installation, ROCM_PATH may be somewhere else. Adjust the path accordingly.

7. Build the PromptForge server

cd "$ROCMFPX"

cmake -S . -B build-promptforge -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER="$ROCM_PATH/llvm/bin/clang" \
  -DCMAKE_CXX_COMPILER="$ROCM_PATH/llvm/bin/clang++" \
  -DCMAKE_HIP_COMPILER="$ROCM_PATH/llvm/bin/clang++" \
  -DCMAKE_PREFIX_PATH="$ROCM_PATH" \
  -DGGML_HIP=ON \
  -DGGML_CUDA=OFF \
  -DGGML_VULKAN=OFF \
  -DGGML_HIP_FORCE_MMQ=ON \
  -DGGML_HIP_GRAPHS=ON \
  -DGGML_HIP_MMQ_MFMA=ON \
  -DGGML_HIP_NO_VMM=ON \
  -DGGML_HIP_ROCWMMA_FATTN=OFF \
  -DGGML_NATIVE=ON \
  -DAMDGPU_TARGETS=gfx1151 \
  -DGPU_BUILD_TARGETS=gfx1151 \
  -DPROMPTFORGE_CK_ROOT="$CK" \
  -DGGML_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_SERVER=ON

Build:

cmake --build build-promptforge \
  --target llama-server \
  -j"$(nproc)"

8. Configure PromptForge for this model

Enable the two model-specific sidecars:

export PROMPTFORGE_SIDECAR="$FFN"
export PROMPTFORGE_GDN_SIDECAR="$GDN"

export PROMPTFORGE_MODE=m2048_fused_tail1476
export PROMPTFORGE_ENABLE_SMALLM_W8=1

export GGML_CUDA_GRAPH_OPT=0
export HIP_VISIBLE_DEVICES=0
export HSA_OVERRIDE_GFX_VERSION=11.5.1

Because this repository does not contain a Heretic-specific Output-K8 proxy, explicitly disable that path:

unset PROMPTFORGE_MTP_OUTPUT_K8
unset PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE
unset PROMPTFORGE_MTP_OUTPUT_K8_PROXY

Also keep the experimental IU4/ngram paths disabled:

unset PROMPTFORGE_IU4_SIDECAR
unset PROMPTFORGE_GDN_IU4_SIDECAR
unset PROMPTFORGE_ENABLE_NGRAM_MOD
unset PROMPTFORGE_ENABLE_NGRAM_M65_IU4

For the pre-Output-K8 MTP selection path:

export LLAMA_MTP_CPU_ARGMAX_FASTPATH=1
unset LLAMA_MTP_BACKEND_GREEDY

Set runtime library paths:

export LD_LIBRARY_PATH="$ROCMFPX/build-promptforge/bin:$ROCM_PATH/lib:$ROCM_PATH/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

9. Start the server

cd "$ROCMFPX"

./build-promptforge/bin/llama-server \
  -m "$MODEL" \
  --alias heretic \
  --host 127.0.0.1 \
  --port 8080 \
  --jinja \
  -fit off \
  -dev ROCm0 \
  -ngl 999 \
  -c 262144 \
  -b 2048 \
  -ub 2048 \
  -fa on \
  -ctk f16 \
  -ctv f16 \
  -t 16 \
  -tb 16 \
  -np 1 \
  -ctxcp 0 \
  --cache-ram 0 \
  --no-cache-prompt \
  --no-cache-idle-slots \
  --timeout 3600 \
  --metrics \
  --spec-type draft-mtp \
  --spec-draft-device ROCm0 \
  --spec-draft-ngl 999 \
  --spec-draft-type-k f16 \
  --spec-draft-type-v f16 \
  --spec-draft-n-max 4 \
  --spec-draft-n-min 0 \
  --spec-draft-p-min 0.0 \
  --spec-draft-p-split 0.10 \
  --spec-draft-backend-sampling \
  --temp 0.7 \
  --top-p 0.95 \
  --reasoning off \
  --reasoning-format none \
  --reasoning-budget -1

The OpenAI-compatible API will be available at:

http://127.0.0.1:8080/v1

10. Test the server

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "heretic",
    "messages": [
      {
        "role": "user",
        "content": "Explain why the sky appears blue."
      }
    ],
    "max_tokens": 256,
    "temperature": 0.7
  }'

Smaller context configurations

The CIRU v3 reference profile uses:

-c 262144

which is appropriate for a high-memory Strix Halo configuration but consumes substantial unified memory.

If you do not need the full context window, you can reduce it, for example:

-c 65536

or:

-c 131072

This does not change the GGUF or PFS files; it only changes runtime memory allocation.

---

GGUF-only operation

The .pfs files are optional.

To run the compact model without PromptForge:

unset PROMPTFORGE_SIDECAR
unset PROMPTFORGE_GDN_SIDECAR
unset PROMPTFORGE_MODE
unset PROMPTFORGE_ENABLE_SMALLM_W8
unset PROMPTFORGE_MTP_OUTPUT_K8
unset PROMPTFORGE_MTP_OUTPUT_K8_PROXY

Then start a compatible llama.cpp / ROCmFPX server with the GGUF normally.

The GGUF remains the authoritative model representation. PromptForge sidecars are alternate compute views used only on qualified execution paths.

---

Vision support

Qwen3.8-27B is natively multimodal, but this repository currently focuses on the language-model GGUF and PromptForge language-model sidecars.

No vision projector is assumed by the launch command above.

If adding multimodal support, use a compatible Qwen3.8 projector whose provenance has been verified against the source checkpoint rather than blindly mixing unrelated model artifacts.

---

Why the PFS files are model-specific

PromptForge does not simply contain generic kernels.

The sidecars contain quantized/prepacked views of the actual model weights.

Therefore:

Heretic GGUF + Heretic PFS     correct

CIRU GGUF + CIRU PFS           correct

Heretic GGUF + CIRU PFS        incorrect

Mixing sidecars between different fine-tunes or ablations may silently cause PromptForge execution to use weights from a different model.

Always keep the GGUF and .pfs files from the same repository/build together.

---

Hardware target

Primary target:

AMD Ryzen AI Max+ 395
Radeon 8060S
gfx1151
Strix Halo

The ROCmFPX formats themselves may work on other AMD hardware, but PromptForge contains specialized model- and kernel-level paths and should not be assumed portable without testing.

CIRU's published Qwen3.8 performance work was developed specifically around gfx1151.

---

Notes on "uncensored"

This repository inherits the behavioral modification of the Heretic-Ara source model.

"Uncensored" or "decensored" describes the model-building goal; it is not a guarantee that every prompt will produce a particular response, nor that the model is more accurate than the original Qwen checkpoint.

Quantization can also introduce small numerical differences relative to BF16.

Users should evaluate the model for their own workloads.

---

Credits

Model:

• Qwen / Qwen3.8-27B
• trohrbaugh / Qwen3.8-27B-heretic-ara
• p-e-w / Heretic

Quantization and inference:

• CIRU ROCmFPX
• CIRU Qwen3.8 ActiveFPX + PromptForge
• llama.cpp
• AMD Composable Kernel

Thanks to the Qwen team, the Heretic contributors, CIRU Inference Lab, llama.cpp contributors, and AMD ROCm/Composable Kernel contributors.

License

The upstream Qwen3.8 model is released under Apache-2.0. This derived model repository follows the applicable upstream model license.

Runtime projects and third-party components retain their own respective licenses.