Downloads · 30 days
16
4% of all-time downloads
shantipriya/odia-ocr-qwen-finetuned
odia-ocr-qwen-finetuned is a image-text-to-text model from shantipriya. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Fine-tuned vision-language model for Odia (Odisha script) OCR based on Qwen/Qwen2.5-VL-3B-Instruct.
Downloads · 30 days
16
4% of all-time downloads
All-time downloads
390
Public
Parameters
3.8B
126 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors17.4 GB · 88%
From the Hugging Face model README
Fine-tuned vision-language model for Odia (Odisha script) OCR based on Qwen/Qwen2.5-VL-3B-Instruct.
| Base model | Qwen/Qwen2.5-VL-3B-Instruct |
| Dataset | shantipriya/odia-ocr-merged (~73 K images) |
| Method | LoRA (r=128, α=256, 7 modules) |
| Epochs | 3 |
| Batch size | 16 (effective, bf16) |
| GPU | NVIDIA H100 80 GB |
| Framework | Transformers + PEFT + TRL |
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel
import torch
from PIL import Image
base = "Qwen/Qwen2.5-VL-3B-Instruct"
adapter = "shantipriya/odia-ocr-qwen-finetuned"
processor = AutoProcessor.from_pretrained(base)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
base, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter)
model.eval()
image = Image.open("odia_text.png").convert("RGB")
messages = [{"role": "user", "content": [
{"type": "image", "image": image},
{"type": "text", "text": "Extract all Odia text from this image. Return only the text."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
with torch.no_grad():
out = model.generate(**inputs, max_new_tokens=512, temperature=0.1, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Apache 2.0