Downloads · 30 days
195
1% of all-time downloads
Qwen3.8-27B-GPTQ-INT8-MTP
shawnw3i/Qwen3.8-27B-GPTQ-INT8-MTP
Qwen3.8-27B-GPTQ-INT8-MTP is a image-text-to-text model from shawnw3i. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
- GPTQ Marlin kernel supported (auto-converted by vLLM at runtime) - MTP speculative decoding supported out of the box - 80+ tok/s on a single A800 80GB (vLLM 0.27.1, MTP enabled, fp8 KV cache) - Vision inputs supported.
All-time downloads
16.3K
Public
Parameters
13.4B
37.3 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors37.2 GB · 100%
Parameter types
How the weights are stored.
I3218.8B · 68%
Try a prompt
Base models
Model card
From the Hugging Face model README
shawnw3i/Qwen3.8-27B-GPTQ-INT8-MTP
Highlights
- GPTQ Marlin kernel supported (auto-converted by vLLM at runtime)
- MTP speculative decoding supported out of the box
- 80+ tok/s on a single A800 80GB (vLLM 0.27.1, MTP enabled, fp8 KV cache)
- Vision inputs supported.
vLLM
vllm serve shawnw3i/Qwen3.8-27B-GPTQ-INT8-MTP \
--max-model-len 65536 \
--reasoning-parser qwen3 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
- Task
- Image-Text-to-Text
- Library
- transformers
- Type
- qwen3_5
- License
- apache-2.0
- Created
- Aug 15, 2026
- Updated
- Aug 15, 2026