Downloads · 30 days
18
2% of all-time downloads
ProCreations/grug-v1.1-qwen-3.8-27b-mtp-awq-int4
grug-v1.1-qwen-3.8-27b-mtp-awq-int4 is a image-text-to-text model from ProCreations. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
vLLM-oriented quantization of ProCreations/grug-v1.1-qwen-3.8-27b-mtp. Activation-aware W4A16 asymmetric INT4 with group size 128, calibrated on 128 Ultrachat samples at 1024 tokens. The trained MTP head remains BF16.…
Downloads · 30 days
18
2% of all-time downloads
All-time downloads
786
Public
Parameters
5.8B
19.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors19.5 GB · 100%
How the weights are stored.
I3224.3B · 88%
From the Hugging Face model README
vLLM-oriented quantization of ProCreations/grug-v1.1-qwen-3.8-27b-mtp. Activation-aware W4A16 asymmetric INT4 with group size 128, calibrated on 128 Ultrachat samples at 1024 tokens. The trained MTP head remains BF16. The vision tower, embeddings, LM head, and precision-sensitive GatedDeltaNet a/b gates also remain BF16.
The quantized backbone comes from ProCreations/grug-v1.1-qwen-3.8-27b-awq-int4 at
df0e54f797a99d980608849be9331ae247920406. This is exact because every one of the 15 backbone BF16 shards in the
normal and MTP source repositories has the same Xet object hash. The trained MTP shard comes
from ProCreations/grug-v1.1-qwen-3.8-27b-mtp@f8d12d372177041c52751f4e2987ccc17fd4f3d9.
vllm serve ProCreations/grug-v1.1-qwen-3.8-27b-mtp-awq-int4 --max-model-len 32768 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
--speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'
The MTP shard is explicitly present in model.safetensors.index.json, and mtp.* is excluded
from the packed quantization scheme so vLLM loads its BF16 weights instead of looking for
weight_packed tensors.