Qwen3.5-4B-ROCmFP4-GGUF

kingjones777/Qwen3.5-4B-ROCmFP4-GGUF

Qwen3.5-4B-ROCmFP4-GGUF is a image-text-to-text model from kingjones777. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.

First ROCmFP4/ROCmFPX quantization of Qwen/Qwen3.5-4B. Four quants, built from the Hub BF16 GGUF (no re-conversion), each individually verified on real hardware.

Downloads · 30 days

176

27% of all-time downloads

All-time downloads

647

Public

Repo size

16 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf16 GB · 100%

At a glance

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Access: Public
Created: Aug 17, 2026
Updated: Aug 17, 2026
SHA: db12711f

Try a prompt

Base models

Qwen/Qwen3.5-4B

Model card

From the Hugging Face model README

Qwen3.5-4B — ROCmFP4 / ROCmFPX GGUF

First ROCmFP4/ROCmFPX quantization of Qwen/Qwen3.5-4B. Four quants, built from the Hub BF16 GGUF (no re-conversion), each individually verified on real hardware.

Built for AMD Strix Halo (gfx1151) — Ryzen AI MAX+ 395, 128 GB unified — with the ROCmFPX llama.cpp fork.

⚠️ Needs a ROCmFPX-capable llama.cpp build. These will not load in stock llama.cpp / Ollama / LM Studio.

Variants

file	ftype	size	`token_embd`	decode	correctness
`Qwen3.5-4B-Q4_0_ROCMFP4_COHERENT.gguf`	102	2.37 GiB	Q6_K	55.57 t/s	3/3
`Qwen3.5-4B-Q6_0_ROCMFPX_AGENT.gguf`	114	3.67 GiB	Q8_0	37.13 t/s	3/3
`Qwen3.5-4B-Q8_0_ROCMFPX.gguf`	111	4.07 GiB	Q8_0	37.25 t/s	3/3
`Qwen3.5-4B-Q8_0_ROCMFPX_AGENT.gguf`	115	4.12 GiB	Q8_0	36.92 t/s	3/3

mmproj-BF16.gguf is included — required for image input (-fa off).

Which to pick: the 4-bit (102) — fastest and smallest, same answers as the 8-bits.

⚠️ Q6_0_ROCMFPX_AGENT (114) is the weakest choice here: larger than the 4-bit and slower than the plain 8-bit. The AGENT recipe raises speculative-draft acceptance, and Qwen3.5-4B ships no drafter, so that benefit cannot apply. It is included for completeness.

Measured — not estimated

Hardware: Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified. -ngl 999 -c 4096 -fa on -fit off. 300 tokens, warm-up discarded, median of 3.

ftype	run 1 / 2 / 3	median	spread
102	55.58 55.57 55.56	55.57	1.0
114	37.15 37.13 37.13	37.13	1.001
111	37.2 37.25 37.41	37.25	1.006
115	36.85 36.92 36.97	36.92	1.003

No MTP/EAGLE drafter ships with this model, so these are the deployed numbers with nothing left on the table.

Head protection

Qwen3.5-4B has tied embeddings — there is no output.weight tensor, so --output-tensor-type is a silent no-op and --token-embedding-type is the only flag that protects the head. Audited by exact tensor name on every artifact.

   2543899552  Qwen3.5-4B-Q4_0_ROCMFP4_COHERENT.gguf
   3935341472  Qwen3.5-4B-Q6_0_ROCMFPX_AGENT.gguf
   4370869152  Qwen3.5-4B-Q8_0_ROCMFPX.gguf
   4426410912  Qwen3.5-4B-Q8_0_ROCMFPX_AGENT.gguf

Credits

Base model: Qwen/Qwen3.5-4B
Quantization stack: ROCmFPX

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Created: Aug 17, 2026
Updated: Aug 17, 2026