Downloads · 30 days
136
25% of all-time downloads
jaehyeono/Qwen3.8-27B-W3A16-GPTQ
Qwen3.8-27B-W3A16-GPTQ is a image-text-to-text model from jaehyeono. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
3-bit weight-quantized version of Qwen/Qwen3.8-27B, produced with GPTQ via llm-compressor.
Downloads · 30 days
136
25% of all-time downloads
All-time downloads
539
Public
Parameters
10.5B
24.5 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors24.5 GB · 100%
How the weights are stored.
I3218.8B · 69%
From the Hugging Face model README
3-bit weight-quantized version of Qwen/Qwen3.8-27B, produced with GPTQ via llm-compressor.
The vision tower is left in bf16 — only the language decoder is quantized. The model remains a fully functional vision-language model; image+text inference works as with the base model.
| Weights | INT3, group size 128, symmetric |
| Activations | bf16 (unquantized) |
| Format | compressed-tensors / pack-quantized |
| Quantized | language decoder (256 Linear modules) |
| Kept in bf16 | vision tower, lm_head, linear-attention projections |
| Size | 55.6 GB → 22.8 GB (~2.4x smaller) |
This model uses the qwen3_5 architecture, which requires transformers >= 5.14:
pip install "transformers>=5.14" compressed-tensors
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image
model_id = "jaehyeono/Qwen3.8-27B-W3A16-GPTQ"
model = AutoModelForImageTextToText.from_pretrained(
model_id, dtype="auto", device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)
messages = [{"role": "user", "content": [
{"type": "image"},
{"type": "text", "text": "Describe this image in one sentence."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[Image.open("example.jpg")], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=64)
print(processor.decode(out[0], skip_special_tokens=True))
Quantized with GPTQModifier from llm-compressor 0.13.0 (compressed-tensors 0.18.0).
| Parameter | Value |
|---|---|
| Bits | 3 |
| Group size | 128 |
| Symmetric | true |
| Strategy | group |
| Activation ordering | static |
| Dampening fraction | 0.01 |
| Observer | memoryless_minmax |
| Calibration | 128 image–caption pairs from Flickr30k |
| Seed | 42 |
The full recipe is included as recipe.yaml.
GPTQ needs activation statistics for every module it touches. The vision tower contributes
a small fraction of total parameters (depth 27, hidden 1152) compared to the language decoder
(64 layers, hidden 5120), so excluding it costs little compression while avoiding degradation
in the visual pathway. Linear-attention projections and lm_head are likewise excluded, per
common practice for these layers.
Apache-2.0, inherited from Qwen/Qwen3.8-27B.