BTL-4-ROCmFP4-STRIX_LEAN-GGUF

BTL-4 — ROCmFP4 STRIX_LEAN (GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

ROCmFP4 quantization of badtheorylabs/BTL-4, built for AMD Strix Halo (gfx1151) with the ROCmFPX fork of llama.cpp.

BTL-4 is a 70.2 GB bf16 MoE vision model — Qwen3_5MoeForConditionalGeneration, 40 layers, 256 experts / 8 active, hidden 2048, vocab 248320. An mmproj is included and is required for image input.

Measured results (Ryzen AI MAX+ 395, 124 GB, gfx1151)

LEAN is −12% on disk and +12% faster than Q4_K_M, at equal-or-better quality. Quality used greedy decoding with the baseline run twice to establish the noise floor (Δ=0 between passes).

A sister build, Q4_0_ROCMFP4_STRIX (4.31 BPW, 17.39 GiB), was produced independently on a second box and measured +11.4% over Q4_K_M at 10/10 = 10/10 quality. The two tiers are within ~67 MB and ~0.02 BPW of each other; prior art on this architecture family favours plain STRIX for quality-critical work.

⚠️ Q3 is a trap on this architecture

Q3 quantizes larger than LEAN. Always run --dry-run and read the real BPW — the advertised value is not reliable.

⛔ No speculative decoding

Upstream BTL-4 ships mtp_num_hidden_layers: 0 — there are no MTP tensors. Do not pass a speculative/MTP flag. A spec flag with no draft tensors loads and reports READY while producing garbage.

Usage

export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1

llama-server \
  -m BTL-4-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-BTL-4-f16.gguf \
  -ngl 999 -fa on -dio --no-warmup --jinja \
  -c 32768 --parallel 1

Requires a ROCmFP4-capable llama.cpp build; stock llama.cpp will not load these tensor types.

License

Inherits the licence of the base model, badtheorylabs/BTL-4.

<!-- PEER-TABLE:START -->

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

Base model: badtheorylabs/BTL-4. Generated from Hub metadata; download counts move over time.

<!-- PEER-TABLE:END --> <!-- CREDITS:START -->

Acknowledgements

This build would not exist without the work below. Please star and follow these projects — the quantisation format used here is their engineering, not mine.

ROCmFPX — maintained by charlie12345 / caf The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork. Every ROCmFP4 file in this repository was produced with its llama-quantize, and runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney, PlunderStruck and Aydan S., and acknowledges AMD for hardware support. Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights, licences and capabilities are theirs. This repository contributes quantisation and measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->