Qwen3.6-35B-A3B-ROCmFP4-STRIX_LEAN-GGUF

gsrunion/Qwen3.6-35B-A3B-ROCmFP4-STRIX_LEAN-GGUF

Qwen3.6-35B-A3B-ROCmFP4-STRIX_LEAN-GGUF is a image-text-to-text model from gsrunion. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.

Q40ROCMFP4STRIXLEAN GGUF quantization of Qwen/Qwen3.6-35B-A3B (Qwen3.5-MoE architecture, ~3B active, multimodal), produced for AMD Strix Halo (Ryzen AI Max, gfx1151).

Downloads · 30 days

102

7% of all-time downloads

All-time downloads

1.4K

Public

Repo size

20.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf19.9 GB · 99%

At a glance

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Access: Public
Created: Jul 23, 2026
Updated: Jul 23, 2026
SHA: f3be5a9c

Try a prompt

Base models

Qwen/Qwen3.6-35B-A3B

Model card

From the Hugging Face model README

Qwen3.6-35B-A3B — ROCmFP4 STRIX_LEAN quant for AMD Strix Halo

Q4_0_ROCMFP4_STRIX_LEAN GGUF quantization of Qwen/Qwen3.6-35B-A3B (Qwen3.5-MoE architecture, ~3B active, multimodal), produced for AMD Strix Halo (Ryzen AI Max, gfx1151).

⚠️ These files use ROCmFP4 tensor types (ggml type 106) — stock llama.cpp will reject them (invalid ggml type). Use either:

kyuz0/amd-strix-halo-toolboxes rocm-7.2.4-rocmfp4 image (public, maintained), or

the Hal0ai/Hal0_ROCmFPX llama.cpp fork.

First published STRIX_LEAN of this model as far as I can tell (a ROCMFP4_FAST and an MTP-focused ROCmFP4 exist elsewhere — the STRIX_LEAN recipe trades a hair of file size for the Strix attn-K/V quality layout + Q5_K token embeddings).

Files

File	Quant	BPW	Size	Notes
`Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf`	Q4_0_ROCMFP4_STRIX_LEAN	4.29	19.0 GB	Fits a 64 GB carve-out with long context to spare
`mmproj-BF16.gguf`	BF16	—	0.90 GB	Vision projector — load alongside the quant
`imatrix_unsloth.gguf_file`	—	—	184 MB	Importance matrix used for calibration (from unsloth)

Measured performance

AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X), ROCm backend, kyuz0 rocm-7.2.4-rocmfp4 toolbox, llama-bench -fa 1 --mmap 0:

test	tok/s
pp512	1102
tg128	63.1

For reference, the same model as UD-Q4_K_XL runs 48 tok/s on identical hardware — the ROCmFP4 kernels are worth ~+31% decode on Strix Halo.

How it was made

BF16 GGUF source and imatrix from unsloth/Qwen3.6-35B-A3B-MTP-GGUF:

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Qwen3.6-35B-A3B-BF16-00001-of-00002.gguf \
  Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

(nextn/MTP metadata from the source survives conversion, though speculative decoding showed no speedup on Strix Halo in my testing — plain decode is recommended.)

Serving

llama-server -m Qwen3.6-35B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-BF16.gguf -ngl 999 -fa on --jinja -c 65536

Companion repos

Credits

Base model: Qwen — Qwen3.6-35B-A3B (Apache-2.0)
BF16 GGUF conversion + imatrix: unsloth
ROCmFP4 quant formats: Hal0ai/Hal0_ROCmFPX; maintained builds: kyuz0

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Languages: en, zh
Created: Jul 23, 2026
Updated: Jul 23, 2026