Qwen2-VL-2B-Instruct-OpenVINO-INT8

Qwen2-VL-2B-Instruct-OpenVINO-INT8 (Silver Series)

This repository contains the Silver Series optimized OpenVINO™ IR version of Qwen2-VL-2B-Instruct, quantized to INT8 precision using NNCF. This high-fidelity multimodal engine is designed for high-accuracy OCR and complex visual reasoning in production environments.

---

🐍 Python Inference (Optimum-Intel)

To run this Silver Series vision engine locally using the optimum-intel library:

from optimum.intel import OVModelForVisualCausalLM
from transformers import AutoProcessor
from PIL import Image

model_id = "CelesteImperia/Qwen2-VL-2B-Instruct-OpenVINO-INT8"
processor = AutoProcessor.from_pretrained(model_id)
model = OVModelForVisualCausalLM.from_pretrained(model_id)

image = Image.open("path/to/your/image.jpg")
prompt = "Transcribe all text from this industrial label with high precision."

inputs = processor(text=[prompt], images=[image], return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0], skip_special_tokens=True))

---

💻 For C# / .NET Users (OpenVINO.GenAI)

The Silver Series vision models are optimized for the native OpenVINO.GenAI framework, enabling ultra-low latency multimodal automation in Windows applications.

using OpenVino.GenAI;

// 1. Initialize the Visual-LLM Pipeline
var device = "CPU"; // Use "GPU" for accelerated dual-GPU inference
using var pipe = new VLMPipeline("path/to/qwen2-vl-int8-model", device);

// 2. Load Visual Input
var image = OpenVino.GenAI.Utils.LoadImage("technical_blueprint.png");
var prompt = "Identify all component serial numbers in this blueprint.";

// 3. Multimodal Execution
var result = pipe.Generate(prompt, image);
Console.WriteLine(result.Texts[0]);

---

🏗️ Technical Details

• Optimization Tool: NNCF (Neural Network Compression Framework)
• Quantization: INT8 Symmetric (Per-channel)
• VRAM Footprint: ~2.5 GB
• Workstation Validation: Dual-GPU (RTX 3090 + RTX A4000)

---

☕ Support the Forge

Maintaining the production line for elite-tier vision models requires significant hardware resources. If these high-fidelity models power your projects, please consider supporting our development:

Scan to support via UPI (India Only):

<img src="https://huggingface.co/datasets/CelesteImperia/Assets/resolve/main/QrCode.jpeg" width="200">

---

📜 License

This model is released under the Apache 2.0 License.

---

Connect with the architect: Abhishek Jaiswal on LinkedIn