Downloads · 30 days
20
13% of all-time downloads
Qwen2.5-VL-7B-Instruct-INT8
sakp7/Qwen2.5-VL-7B-Instruct-INT8
Qwen2.5-VL-7B-Instruct-INT8 is a machine learning model from sakp7. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is an 8-bit quantized version of Qwen2.5-VL-7B-Instruct for more efficient deployment.
All-time downloads
155
Public
Parameters
8.3B
9.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors9.4 GB · 100%
Parameter types
How the weights are stored.
I87.2B · 87%
Model card
From the Hugging Face model README
Qwen2.5-VL-7B-Instruct-INT8
This is an 8-bit quantized version of Qwen2.5-VL-7B-Instruct for more efficient deployment.
Model Details
- Base Model: Qwen/Qwen2.5-VL-7B-Instruct
- Quantization: 8-bit quantization using bitsandbytes
- vLLM Compatible: Yes
- Use Case: Vision-language tasks with reduced memory requirements
Usage with vLLM
from vllm import LLM, SamplingParams
from PIL import Image
# Initialize the model
llm = LLM(model="sakp7/Qwen2.5-VL-7B-Instruct-INT8", quantization="8bit")
# Prepare image and prompt
image = Image.open("example.jpg")
prompt = "Describe this image in detail."
# Generate response
sampling_params = SamplingParams(temperature=0.7, max_tokens=256)
outputs = llm.generate_for_multimodal_prompts([[prompt, image]], sampling_params)
print(outputs[0].outputs[0].text)
- Type
- qwen2_5_vl
- Created
- Apr 3, 2025
- Updated
- Apr 3, 2025