Qwen3.5-2B-ROCmFP4-GGUF

kingjones777/Qwen3.5-2B-ROCmFP4-GGUF

Qwen3.5-2B-ROCmFP4-GGUF is a image-text-to-text model from kingjones777. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.

First ROCmFP4/ROCmFPX quantization of Qwen/Qwen3.5-2B. Four quants, built from the Hub BF16 GGUF (no re-conversion), each individually verified on real hardware.

Downloads · 30 days

119

24% of all-time downloads

All-time downloads

499

Public

Repo size

7.6 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf7.6 GB · 100%

At a glance

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Access: Public
Created: Aug 17, 2026
Updated: Aug 17, 2026
SHA: be5af690

Try a prompt

Base models

Qwen/Qwen3.5-2B

Model card

From the Hugging Face model README

Qwen3.5-2B — ROCmFP4 / ROCmFPX GGUF

First ROCmFP4/ROCmFPX quantization of Qwen/Qwen3.5-2B. Four quants, built from the Hub BF16 GGUF (no re-conversion), each individually verified on real hardware.

Built for AMD Strix Halo (gfx1151) — Ryzen AI MAX+ 395, 128 GB unified — with the ROCmFPX llama.cpp fork.

⚠️ Needs a ROCmFPX-capable llama.cpp build. These will not load in stock llama.cpp / Ollama / LM Studio.

Variants

file	ftype	size	`token_embd`	decode	correctness
`Qwen3.5-2B-Q4_0_ROCMFP4_COHERENT.gguf`	102	1.12 GiB	Q6_K	106.44 t/s	3/3
`Qwen3.5-2B-Q6_0_ROCMFPX_AGENT.gguf`	114	1.68 GiB	Q8_0	75.05 t/s	3/3
`Qwen3.5-2B-Q8_0_ROCMFPX.gguf`	111	1.83 GiB	Q8_0	77.38 t/s	3/3
`Qwen3.5-2B-Q8_0_ROCMFPX_AGENT.gguf`	115	1.85 GiB	Q8_0	76.77 t/s	3/3

mmproj-BF16.gguf is included — required for image input (-fa off).

Which to pick: the 4-bit (102) — fastest and smallest, same answers as the 8-bits.

⚠️ Q6_0_ROCMFPX_AGENT (114) is the weakest choice here: larger than the 4-bit and slower than the plain 8-bit. The AGENT recipe raises speculative-draft acceptance, and Qwen3.5-2B ships no drafter, so that benefit cannot apply. It is included for completeness.

Measured — not estimated

Hardware: Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified. -ngl 999 -c 4096 -fa on -fit off. 300 tokens, warm-up discarded, median of 3.

ftype	run 1 / 2 / 3	median	spread
102	107.0 106.44 105.46	106.44	1.015
114	76.17 75.05 74.83	75.05	1.018
111	77.38 77.78 76.92	77.38	1.011
115	76.77 76.47 77.04	76.77	1.007

No MTP/EAGLE drafter ships with this model, so these are the deployed numbers with nothing left on the table.

Head protection

Qwen3.5-2B has tied embeddings — there is no output.weight tensor, so --output-tensor-type is a silent no-op and --token-embedding-type is the only flag that protects the head. Audited by exact tensor name on every artifact.

   1202487328  Qwen3.5-2B-Q4_0_ROCMFP4_COHERENT.gguf
   1807376416  Qwen3.5-2B-Q6_0_ROCMFPX_AGENT.gguf
   1969115168  Qwen3.5-2B-Q8_0_ROCMFPX.gguf
   1991397408  Qwen3.5-2B-Q8_0_ROCMFPX_AGENT.gguf

Credits

Base model: Qwen/Qwen3.5-2B
Quantization stack: ROCmFPX

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Created: Aug 17, 2026
Updated: Aug 17, 2026