tamil-ocr-qwen25vl-lora

Tamil OCR — Qwen2.5-VL-3B LoRA Adapter

LoRA adapter fine-tuned on Qwen2.5-VL-3B-Instruct for Tamil optical character recognition. Attach to base model for inference. Merged model: sair390/tamil-ocr-qwen25vl

Model Details

Training Hyperparameters

Training Data

Synthetic data augmentations: blur, noise, rotation, brightness/contrast, JPEG artifacts, erosion, shadow overlays Real data: ~150 Tamil books, ~75K pages Fonts: NotoSerifTamil, NotoSansTamil, MuktaMalar, HindMadurai, Catamaran Text sources: Tirukkural, Wikipedia sentences, digitized book corpus Text layouts: 75% single-line, 25% multi-line Font sizes: 24–72pt (single-line), 24–48pt (multi-line)

Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
from PIL import Image
import torch

# Load base model (4-bit quantized)
base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "unsloth/qwen2.5-vl-3b-instruct-unsloth-bnb-4bit",
    load_in_4bit=True,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained("unsloth/qwen2.5-vl-3b-instruct-unsloth-bnb-4bit")

# Attach LoRA adapter
model = PeftModel.from_pretrained(base_model, "sair390/tamil-ocr-qwen25vl-lora")

# Inference — use temperature=0 for deterministic OCR output
image = Image.open("tamil_page.jpg")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "Read the Tamil text in this image."},
    ],
}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0,        # deterministic — best for OCR
        do_sample=False,
    )

result = processor.decode(output[0], skip_special_tokens=True)
print(result)

4-Stage Inference Pipeline

1. Preprocessing — deskew + adaptive threshold binarization
2. Layout analysis — contour detection, bounding box merge, top-to-bottom sort
3. OCR — Qwen2.5-VL with this LoRA adapter
4. Post-correction — Tamil vowel sign fixes, Unicode filter, common OCR confusions

Inference Requirements

• VRAM: ~6–10 GB (4-bit quantized base)
• Dependencies: transformers, peft, torch, Pillow

Limitations

• Optimized for printed Tamil text; handwritten text not tested
• Performance on degraded/low-resolution scans may vary
• Early checkpoint; full training run in progress