Downloads · 30 days
301
8% of all-time downloads
vadikulous-ai/Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF
Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP-GGUF is a image-text-to-text model from vadikulous-ai. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains the Qwen-AgentWorld-35B-A3B model in UD-Q4KXL quantization with Multi-Token Prediction (MTP) support for speculative decoding.
Downloads · 30 days
301
8% of all-time downloads
All-time downloads
3.9K
Public
Repo size
50.9 GB
Likes
5
Public
Click a slice to open those files.
.gguf50.9 GB · 100%
From the Hugging Face model README
This repository contains the Qwen-AgentWorld-35B-A3B model in UD-Q4_K_XL quantization with Multi-Token Prediction (MTP) support for speculative decoding.
The MTP heads from Qwen3.6-35B-A3B were extracted and grafted onto the Qwen-AgentWorld-35B-A3B model. Both models share the same architecture (qwen35moe), making this transfer possible. The resulting model includes all base layers from AgentWorld plus the MTP prediction head from the original Qwen3.6 model.
Performance results were achieved on AMD Strix Halo:
The high acceptance rate indicates that the MTP heads from the base model transfer well to the AgentWorld fine-tune.
./llama-server \
-m Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf \
-ngl 99 -c 8192 -fa on \
--spec-type draft-mtp \
--spec-draft-n-max 3
For multimodal capabilities with reasoning and optimized parameters:
MODEL_PATH="Qwen-AgentWorld-35B-A3B-UD-Q4_K_XL-MTP.gguf"
MMPROJ_PATH="mmproj-BF16.gguf"
./llama-server \
--model "$MODEL_PATH" \
--mmproj "$MMPROJ_PATH" \
-ngl 99 \
--host 0.0.0.0 \
--port 8000 \
--ctx-size 262144 \
--reasoning "on" \
--kv-unified \
-np 2 \
-fa "on" \
--no-mmap \
--image-min-tokens 1024 \
--spec-type "draft-mtp" \
--spec-draft-n-max 3 \
--spec-draft-p-min 0.75 \
-b 2048 \
-ub 512
Parameters:
--mmproj: Multimodal projector for vision capabilities--reasoning "on": Enable reasoning mode--kv-unified: Unified KV cache-np 2: 2 parallel slots--spec-type "draft-mtp": Enable MTP speculative decoding--spec-draft-n-max 3: Generate up to 3 draft tokens per step--spec-draft-p-min 0.75: Minimum probability threshold for draft tokens-b 2048: Batch size-ub 512: U-batch size