Qwen3.5-2B-MTP-fast-GGUF

petr567/Qwen3.5-2B-MTP-fast-GGUF

Qwen3.5-2B-MTP-fast-GGUF is a image-text-to-text model from petr567. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.

Fast MTP profiles for Bartowski's Qwen3.5-2B Q4KM GGUF in LM Studio and llama.cpp Vulkan.

Downloads · 30 days

135

16% of all-time downloads

All-time downloads

866

Public

Repo size

2.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf2.1 GB · 100%

At a glance

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Access: Public
Created: Aug 2, 2026
Updated: Aug 2, 2026
SHA: 72fdc6b6

Try a prompt

Base models

Qwen/Qwen3.5-2B

Model card

From the Hugging Face model README

Qwen3.5-2B MTP fast

Fast MTP profiles for Bartowski's Qwen3.5-2B Q4_K_M GGUF in LM Studio and llama.cpp Vulkan.

The GGUF weights are unchanged. The speed-up is enabled by the included LM Studio MTP load profile.

Files

File	Description
`Qwen_Qwen3.5-2B-Q4_K_M.gguf`	Q4_K_M model with built-in MTP heads
`mmproj-Qwen_Qwen3.5-2B-f16.gguf`	Vision projector
`load_profile.ps1`	LM Studio baseline/fast profile loader
`benchmark-summary.json`	Machine-readable benchmark summary
`benchmark-summary.csv`	Compact context-speed table
`cloudpersonal-benchmark-summary.json`	Strix Halo server benchmark summary
`cloudpersonal-benchmark.csv`	Strix Halo context-speed table

LM Studio fast profile

LM Studio 0.4.20 or newer
context: up to 262,144 tokens
parallel: 1
Flash Attention: on
KV cache GPU offload: on
MTP: on
maximum draft tokens: 2
minimum continue probability: 0.20

Start LM Studio's local server on port 1234, then run:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\load_profile.ps1 -Profile ornith -ContextLength 262144

RTX 4060 Laptop measurements

Mean decode speed, three runs per context, 128 generated tokens:

Prompt context	MTP off	MTP fast	Change
1K	128.86 tok/s	177.92 tok/s	+38.08%
4K	126.74 tok/s	175.16 tok/s	+38.20%
8K	122.89 tok/s	172.74 tok/s	+40.57%
16K	117.35 tok/s	165.21 tok/s	+40.78%
32K	106.28 tok/s	148.93 tok/s	+40.13%
64K	88.71 tok/s	132.06 tok/s	+48.86%
128K	67.85 tok/s	105.60 tok/s	+55.64%
250K	48.27 tok/s	76.82 tok/s	+59.14%

maximum single run: 179.09 tok/s
maximum repeated mean: 177.92 tok/s
geometric mean speed-up over all contexts: 1.4497x
maximum loaded context: 262,144 tokens

Test system: Windows 11, Intel Core i7-13650HX, 63.74 GiB RAM, NVIDIA RTX 4060 Laptop 8 GiB, LM Studio 0.4.20, llama.cpp CUDA runtime 2.27.1.

Ryzen AI MAX+ 395 / Radeon 8060S server measurements

Direct llama.cpp b9994 Vulkan A/B on the same GGUF and prompts. The server used a 262,144-token window, Q8_0 K/V cache, Flash Attention, full GPU offload, parallel 1, and three runs per context. Other resident AI services were idle during the test.

Prompt context	MTP off	MTP fast	Change
1K	125.68 tok/s	203.82 tok/s	+62.18%
4K	123.79 tok/s	197.10 tok/s	+59.22%
8K	121.23 tok/s	188.57 tok/s	+55.55%
16K	115.64 tok/s	180.21 tok/s	+55.84%
32K	108.47 tok/s	159.48 tok/s	+47.02%
64K	94.00 tok/s	131.15 tok/s	+39.52%
128K	75.76 tok/s	95.64 tok/s	+26.24%
250K	56.06 tok/s	63.67 tok/s	+13.59%

maximum single run: 205.20 tok/s
maximum repeated mean: 203.82 tok/s
geometric mean speed-up over all contexts: 1.4393x
deterministic output hashes matched at every context

Server: AMD Ryzen AI MAX+ 395 with Radeon 8060S, 128 GiB UMA, Ubuntu, llama.cpp b9994 (14d3ba45f), Vulkan.

Credits

Base model: Qwen/Qwen3.5-2B
GGUF quantization: bartowski/Qwen_Qwen3.5-2B-GGUF

Task: Image-Text-to-Text
Library: gguf
License: apache-2.0
Created: Aug 2, 2026
Updated: Aug 2, 2026