Qwen2.5-VL-3B-Instruct-int8-ov

Qwen2.5-VL-3B-Instruct — OpenVINO INT8

INT8 OpenVINO export of Qwen/Qwen2.5-VL-3B-Instruct for Intel GPUs. INT8 retains fine detail (OCR, small numbers) better than INT4 — the proven small-VLM workhorse in NoLlama, where this exact conversion recipe has run vision workloads since mid-2026.

⚠️ License: Qwen Research License (non-commercial)

Unlike its 7B sibling (Apache-2.0), the 3B base model is under Alibaba's Qwen Research License — research and evaluation use only; commercial use requires a license from Alibaba Cloud. This conversion inherits those terms (see LICENSE). Modification notice: weights converted to OpenVINO IR and quantized to INT8 with the toolchain below; no other changes.

Export command

optimum-cli export openvino --model Qwen/Qwen2.5-VL-3B-Instruct \
  --weight-format int8 --trust-remote-code <output_dir>

Toolchain: OpenVINO 2026.1.0, optimum-intel 1.27.0, transformers 4.57.6. Runs on openvino-genai VLMPipeline (GPU recommended; streaming works as of genai 2026.1).

Usage

import numpy as np, openvino as ov, openvino_genai as og
from PIL import Image
pipe = og.VLMPipeline("Qwen2.5-VL-3B-Instruct-int8-ov", "GPU")
img = ov.Tensor(np.array(Image.open("photo.jpg").convert("RGB")))
print(pipe.generate("What is in this image?", image=img,
                    generation_config=og.GenerationConfig(max_new_tokens=128)))

Or with NoLlama: python nollama.py --gpu-model-dir ~/models/Qwen2.5-VL-3B-Instruct-int8-ov. VLM prompting tip from NoLlama's docs: keep prompts dead simple for a 3B — one question, one answer; put the logic in your code, not the prompt.