sinhala-lightonocr-2-1b-Qlora

Sinhala LightOnOCR-2-1B QLoRA Model 🇱🇰

<div align="center">

Fine-tuned LightOnOCR-2-1B model for high-accuracy Sinhala language OCR on historical legal documents

🚀 Quick Start • 📊 Performance • 📖 Usage • 🔧 Training • 🎓 Citation

</div>

---

📋 Model Description

This model is a QLoRA fine-tuned version of LightOnOCR-2-1B specifically optimized for Sinhala (සිංහල) language OCR on historical and contemporary legal documents. The model achieves 98.95% character accuracy on a test set spanning over a century of Sri Lankan legal texts (1981-2019).

Key Features

• 🎯 High Accuracy: 98.95% character accuracy on Sinhala legal documents
• 📜 Historical Coverage: Evaluated on documents from 1981-2019
• ⚡ Efficient: QLoRA fine-tuning with 4-bit quantization (~3.67% trainable parameters)
• 🖥️ Optimized: Trained on NVIDIA RTX 4080 SUPER
• 💾 Low Resource: Runs on consumer GPUs with 4-bit quantization
• 🔄 Flexible Loading: Supports both QLoRA (4-bit) and standard LoRA (full-precision) inference

Model Details

---

📊 Performance Metrics

Overall Performance (202 Test Samples)

Summary Statistics

---

🚀 Quick Start

Installation

pip install transformers==5.0.0 peft bitsandbytes Pillow

Option 1: QLoRA Inference (4-bit Quantized — Recommended for Low VRAM)

Load the base model with 4-bit quantization and apply the LoRA adapter on top. This matches the original training setup and requires ~2-3 GB VRAM.

import torch
from transformers import LightOnOcrForConditionalGeneration, LightOnOcrProcessor, BitsAndBytesConfig
from peft import PeftModel
from PIL import Image

# Configuration
BASE_MODEL_ID = "lightonai/LightOnOCR-2-1B"
ADAPTER_ID = "avishadilhara/sinhala-lightonocr-2-1b-Qlora"
LONGEST_EDGE = 1540

# Load processor
processor = LightOnOcrProcessor.from_pretrained(ADAPTER_ID)
processor.tokenizer.padding_side = "left"

# Load base model with 4-bit quantization (QLoRA)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4"
)

model = LightOnOcrForConditionalGeneration.from_pretrained(
    BASE_MODEL_ID,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    quantization_config=bnb_config
)

# Load QLoRA adapter
model = PeftModel.from_pretrained(model, ADAPTER_ID)
model.eval()

# Run inference
image = Image.open("your_image.png").convert("RGB")

messages = [
    {"role": "user", "content": [{"type": "image"}]},
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(
    text=text,
    images=[image],
    return_tensors="pt",
    size={"longest_edge": LONGEST_EDGE},
).to(model.device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=4096,
        do_sample=False,
    )

result = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(result)

Option 2: LoRA Inference (Full Precision — Higher Quality)

Load the base model in full precision (bf16) and apply the LoRA adapter. No quantization — better quality but requires ~4-5 GB VRAM.

import torch
from transformers import LightOnOcrForConditionalGeneration, LightOnOcrProcessor
from peft import PeftModel
from PIL import Image

# Configuration
BASE_MODEL_ID = "lightonai/LightOnOCR-2-1B"
ADAPTER_ID = "avishadilhara/sinhala-lightonocr-2-1b-Qlora"
LONGEST_EDGE = 1540

# Load processor
processor = LightOnOcrProcessor.from_pretrained(ADAPTER_ID)
processor.tokenizer.padding_side = "left"

# Load base model in full precision (no quantization)
model = LightOnOcrForConditionalGeneration.from_pretrained(
    BASE_MODEL_ID,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

# Load LoRA adapter (same weights, no quantization on base)
model = PeftModel.from_pretrained(model, ADAPTER_ID)
model.eval()

# Run inference
image = Image.open("your_image.png").convert("RGB")

messages = [
    {"role": "user", "content": [{"type": "image"}]},
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(
    text=text,
    images=[image],
    return_tensors="pt",
    size={"longest_edge": LONGEST_EDGE},
).to(model.device)

with torch.no_grad():
    generated_ids = model.generate(
        **inputs,
        max_new_tokens=4096,
        do_sample=False,
    )

result = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(result)

Note: Both options use the same LoRA adapter weights. The difference is whether the base model is quantized (QLoRA) or loaded in full precision (LoRA). QLoRA uses less VRAM; LoRA may give slightly better quality.

---

🔧 Training Details

Dataset

Dataset: avishadilhara/sinhala-ocr-lk-acts-1010

QLoRA Configuration

Training Arguments

Training Loss

Best model selected at epoch 3 (lowest validation loss).

Hardware

• GPU: NVIDIA RTX 4080 SUPER
• Training Time: ~3 hours (4 epochs)

---

🎓 Citation

If you use this model, please cite:

@misc{dilhara2026crosstemporalsinhalaocrpagelevel,
      title={Cross-Temporal Sinhala OCR: Page-Level Adaptation and Diachronic Analysis}, 
      author={Avisha Dilhara and Nevidu Jayatilleke},
      year={2026},
      eprint={2606.29378},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={[https://arxiv.org/abs/2606.29378](https://arxiv.org/abs/2606.29378)}
}