Downloads · 30 days
37
3% of all-time downloads
jamiefutch/Qwen3.5-122B-A10B-MXFP4_MOE-MTP-GGUF
Qwen3.5-122B-A10B-MXFP4_MOE-MTP-GGUF is a text generation model from jamiefutch. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
This is a community-merged GGUF build of Qwen3.5-122B-A10B MXFP4MOE with an injected MTP donor layer for llama.cpp MTP speculative decoding.
Downloads · 30 days
37
3% of all-time downloads
All-time downloads
1.1K
Public
Repo size
69.9 GB
Likes
7
Public
Click a slice to open those files.
.gguf69.9 GB · 100%
From the Hugging Face model README
This is a community-merged GGUF build of Qwen3.5-122B-A10B MXFP4_MOE with an injected MTP donor layer for llama.cpp MTP speculative decoding.
This repository is not an official release from Qwen, Unsloth, or LaikaFramework.
This is an experimental community merge. Please report loading issues, MTP compatibility problems, or runtime results in the Discussions tab.
This repository provides a merged GGUF file:
Qwen3.5-122B-A10B-MXFP4_MOE-MTP-merged.gguf
The model is intended for testing with a custom llama.cpp build that supports MTP speculative decoding.
This file was produced from the following sources:
Qwen/Qwen3.5-122B-A10Bunsloth/Qwen3.5-122B-A10B-GGUF, MXFP4_MOELaikaFramework/Qwen3.5-122B-A3B-MTP-Donorconvert.py merge <single-base.gguf> <donor.gguf> <output.gguf>llama.cpp build with MTP support, PR #22673 styleAt the time of testing, complete Qwen3.5-122B-A10B GGUF files with working MTP support were not widely available.
This merge adds the missing MTP/NextN layer metadata and tensors to an existing MXFP4_MOE GGUF base.
The key metadata expected after merge is:
general.architecture = qwen35moe
qwen35moe.block_count = 49
qwen35moe.nextn_predict_layers = 1
Local test environment:
OS: Ubuntu 24.04
Hardware: AMD Ryzen AI Max+ 395 / Radeon 8060S
Memory: 128GB unified memory
Backend: llama.cpp Vulkan / RADV
Context tested: 262144
MTP setting: --spec-type mtp --spec-draft-n-max 2
On the above AMD/Vulkan system, the model successfully loaded as:
qwen35moe_mtp
Observed local performance:
1200-token test:
decode speed: around 31 tok/s
MTP draft acceptance: around 72%
3000-token test:
decode speed: around 31 tok/s
MTP draft acceptance: around 76%
These numbers are only local observations. Your performance may vary significantly depending on backend, hardware, memory bandwidth, context size, and llama.cpp version.
Example command:
LLAMA="/path/to/llama-server"
MODEL="/path/to/Qwen3.5-122B-A10B-MXFP4_MOE-MTP-merged.gguf"
"$LLAMA" \
-m "$MODEL" \
--host 0.0.0.0 \
--port 1236 \
--alias qwen35-122b-mxfp4-mtp \
-ngl 999 \
-c 262144 \
-np 1 \
-b 1024 \
-ub 256 \
--cache-type-k q8_0 \
--cache-type-v q8_0 \
--jinja \
--reasoning off \
--reasoning-format deepseek \
--temp 0.3 \
--top-p 0.95 \
--top-k 20 \
--spec-type mtp \
--spec-draft-n-max 2
For lower KV cache memory usage at very long context, you may test:
--cache-type-k q4_0 \
--cache-type-v q4_0
In local testing, q4 KV reduced memory pressure and remained usable, but q8 KV is recommended as the default quality-first configuration.
A SHA256 checksum is included in SHA256SUMS.
To verify after download:
sha256sum -c SHA256SUMS
The upstream Qwen model is released under Apache-2.0.
This repository follows Apache-2.0 and retains attribution to the upstream projects.
Please also review and respect the license terms of the upstream model and source repositories.