Downloads · 30 days
13
4% of all-time downloads
Qwen2.5-VL-3B-Instruct-GPTQ-Int4
Azaz666/Qwen2.5-VL-3B-Instruct-GPTQ-Int4
Qwen2.5-VL-3B-Instruct-GPTQ-Int4 is a machine learning model from Azaz666. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This is a GPTQ (4-bit) quantized version of Qwen/Qwen2.5-VL-3B-Instruct.
All-time downloads
305
Public
Parameters
3.8B
13.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.4 GB · 100%
Parameter types
How the weights are stored.
I322.8B · 74%
Base models
Model card
From the Hugging Face model README
Qwen__Qwen2.5-VL-3B-Instruct__gptq_int4
This is a GPTQ (4-bit) quantized version of Qwen/Qwen2.5-VL-3B-Instruct.
Quantization Details
- Method: GPTQ
- Bits: 4
- Base model: Qwen/Qwen2.5-VL-3B-Instruct
- Group size: 128
- Quantized portion: LLM backbone only
- Vision tower: FP16 (unchanged)
- Loading: AutoModelForImageTextToText / AutoModelForCausalLM (trust_remote_code where applicable)
Usage
from transformers import AutoProcessor, AutoModelForImageTextToText
import torch
model = AutoModelForImageTextToText.from_pretrained(
"{REPO_ID}",
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("{REPO_ID}", trust_remote_code=True)
Replace {REPO_ID} with the repo ID of this model.
Original Model
See Qwen/Qwen2.5-VL-3B-Instruct for the original FP16 model.
- Type
- qwen2_5_vl
- License
- apache-2.0
- Created
- Mar 22, 2026
- Updated
- Apr 28, 2026