Downloads · 30 days
6.4K
3% of all-time downloads
Qwen3.8-27B-AWQ-MTP
shawnw3i/Qwen3.8-27B-AWQ-MTP
Qwen3.8-27B-AWQ-MTP is a image-text-to-text model from shawnw3i. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- AWQ Marlin kernel supported (auto-converted by vLLM at runtime) - MTP speculative decoding supported out of the box - 110+ tok/s on a single A800 80GB (vLLM 0.27.1, MTP enabled, fp8 KV cache) - Vision inputs supported.
All-time downloads
190K
Public
Parameters
6.5B
33.7 GB on disk
Likes
11
Public
Repo makeup
Click a slice to open those files.
.safetensors20 GB · 100%
Parameter types
How the weights are stored.
I3224.3B · 88%
Try a prompt
Base models
Model card
From the Hugging Face model README
shawnw3i/Qwen3.8-27B-AWQ-MTP
Highlights
- AWQ Marlin kernel supported (auto-converted by vLLM at runtime)
- MTP speculative decoding supported out of the box
- 110+ tok/s on a single A800 80GB (vLLM 0.27.1, MTP enabled, fp8 KV cache)
- Vision inputs supported.
Update
09-27-2026 Because Qwen3.8 utilizes non-traditional attention, the group size was reduced to 64 to prevent precision collapse.
vLLM
vllm serve shawnw3i/Qwen3.8-27B-AWQ-MTP \
--max-model-len 65536 \
--reasoning-parser qwen3 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Aug 14, 2026
- Updated
- Aug 28, 2026