Qwen3-VL-2B-Instruct-ov-int4

Qwen3-VL-2B-Instruct OpenVINO INT4

This repository contains a local OpenVINO IR export of Qwen/Qwen3-VL-2B-Instruct converted for OpenVINO GenAI inference and compressed to INT4 weights.

Requirements

This model is intended for OpenVINO GenAI VLM inference.

pip install -U openvino openvino-tokenizers openvino-genai pillow numpy

Download

hf download shawnxhong/Qwen3-VL-2B-Instruct-ov-int4 --local-dir Qwen3-VL-2B-Instruct-ov-int4

Included files

• OpenVINO GenAI tokenizer and detokenizer
• OpenVINO language model
• OpenVINO text embeddings model
• OpenVINO vision embeddings model
• OpenVINO vision merger and positional embedding models

Minimal local inference

from pathlib import Path

import numpy as np
import openvino as ov
import openvino_genai
from PIL import Image

model_dir = Path("Qwen3-VL-2B-Instruct-ov-int4")
image_path = Path("example.png")

pipe = openvino_genai.VLMPipeline(str(model_dir), "CPU")

image = Image.open(image_path).convert("RGB")
image_tensor = ov.Tensor(np.array(image)[None])

generation_config = pipe.get_generation_config()
generation_config.max_new_tokens = 128
generation_config.do_sample = False

result = pipe.generate(
    "Read the image and return the visible text.",
    image_tensor,
    generation_config,
)

print(result.texts[0])

Minimal script

import openvino as ov
import openvino_genai
from PIL import Image
import numpy as np

pipe = openvino_genai.VLMPipeline("Qwen3-VL-2B-Instruct-ov-int4", "CPU")
image = ov.Tensor(np.array(Image.open("example.png").convert("RGB"))[None])
result = pipe.generate("Read the image and return the visible text.", image, max_new_tokens=128)
print(result.texts[0])

Benchmark with OpenVINO GenAI llm_bench

If you have the openvino.genai source tree locally:

python tools/llm_bench/benchmark.py \
  -m Qwen3-VL-2B-Instruct-ov-int4 \
  -d CPU \
  -t visual_text_gen \
  -i example.png \
  -p "Read the image and return the visible text." \
  -n 3 \
  -ic 128 \
  -r benchmark_report.csv \
  -rj benchmark_report.json

Notes

• This export targets OpenVINO GenAI VLMPipeline.
• The model was converted from Qwen/Qwen3-VL-2B-Instruct to OpenVINO IR and compressed to INT4.
• For OCR-style usage, short prompts like Read the image and return the visible text. work well as a baseline.