Qari-OCR-Fine-Tuned-Kitab-Benchmark

Qari-OCR-Fine-Tuned-Kitab-Benchmark

Model Description

This model is a LoRA fine-tuned version of NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct specifically optimized for Arabic OCR tasks using the comprehensive KITAB-Bench dataset.

Model Details

• Base Model: NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct
• Model Type: Vision-Language Model with LoRA fine-tuning
• Language: Arabic (primary), with multilingual capabilities
• License: [Specify license]
• Fine-tuned for: Arabic Optical Character Recognition (OCR)

Training Configuration

• Training Method: LoRA (Low-Rank Adaptation)
• LoRA Parameters:
  • Rank (r): 16
  • Alpha: 32
  • Dropout: 0.05
  • Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
• Training Epochs: 5
• Batch Size: 4 per device
• Learning Rate: 2e-4
• Optimizer: AdamW 8-bit
• Max Sequence Length: 2048

Dataset

The model was trained on a curated subset of Arabic OCR datasets comprising 3,760 total samples from 13 domain-specific datasets:

Training Data Composition

Total Combined Dataset: 3,760 samples

• Training Set: 3,572 samples (95% of total)
• Held-out Test Set: 188 samples (5% of total)

Source Datasets Used:

• ahmedheakl/arocrbench_synthesizear: 500 samples
• ahmedheakl/arocrbench_patsocr: 500 samples
• ahmedheakl/arocrbench_historyar: 200 samples
• ahmedheakl/arocrbench_historicalbooks: 10 samples
• ahmedheakl/arocrbench_khattparagraph: 200 samples
• ahmedheakl/arocrbench_adab: 200 samples
• ahmedheakl/arocrbench_muharaf: 200 samples
• ahmedheakl/arocrbench_onlinekhatt: 200 samples
• ahmedheakl/arocrbench_khatt: 200 samples
• ahmedheakl/arocrbench_isippt: 500 samples
• ahmedheakl/arocrbench_arabicocr: 50 samples
• ahmedheakl/arocrbench_hindawi: 200 samples
• ahmedheakl/arocrbench_evarest: 800 samples

Data Split

• 95% training (3,572 samples)
• 5% held-out test (188 samples) for final evaluation

Domain Coverage

• Handwritten Text: Historical manuscripts, personal notes, traditional calligraphy
• Printed Text: Books, newspapers, academic papers, legal documents
• Scene Text: Street signs, advertisements, natural environments
• Structured Documents: Tables, forms, layouts
• Historical Documents: Ancient texts, heritage manuscripts
• Synthetic Data: Generated text for augmentation

Performance

Evaluation Results on Held-Out Test Set

Intended Use

Primary Use Cases

• Arabic document digitization
• Historical manuscript transcription
• Multi-domain Arabic text recognition
• RAG (Retrieval-Augmented Generation) document processing pipelines
• Academic research in Arabic NLP and OCR

Direct Use

from transformers import AutoProcessor, AutoModelForVision2Seq
from PIL import Image

# Load model and processor
processor = AutoProcessor.from_pretrained("FatimahEmadEldin/Qari-OCR-Fine-Tuned-Kitab-Benchmark")
model = AutoModelForVision2Seq.from_pretrained("FatimahEmadEldin/Qari-OCR-Fine-Tuned-Kitab-Benchmark")

# Process image
image = Image.open("arabic_document.jpg")
prompt = "Below is the image of one page of a document. Please provide the plain text representation of this document as if you were reading it naturally, ensuring high accuracy."

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": prompt},
        ],
    }
]

text_prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=text_prompt, images=image, return_tensors="pt")

# Generate
generated_ids = model.generate(**inputs, max_new_tokens=2048)
generated_ids = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
response = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

Limitations and Considerations

Known Limitations

• Complex Fonts: Performance may vary with highly stylized or decorative Arabic fonts
• Numeral Recognition: Some challenges with mixed Arabic-Indic numeral systems
• Word Elongation: Handling of kashida (Arabic text elongation) requires improvement
• PDF-to-Markdown: Limited accuracy (best models achieve ~65% on complex layouts)

Bias and Fairness

• Trained primarily on Modern Standard Arabic; dialectical variations may have reduced accuracy
• Historical document performance depends on manuscript quality and preservation state
• Geographic bias toward Gulf and Levantine Arabic text styles

Technical Specifications

Hardware Requirements

• Minimum: 8GB GPU memory for inference
• Recommended: 16GB+ GPU memory for optimal performance
• Training: Conducted on NVIDIA A100 GPUs

Software Dependencies

• transformers >= 4.51.3
• torch >= 2.4.0
• unsloth (for efficient training)
• Pillow for image processing

Training Details

Training Infrastructure

• Framework: Unsloth for efficient LoRA training
• Quantization: 4-bit quantization for memory efficiency
• Mixed Precision: BF16/FP16 based on hardware support

Data Processing

• Images processed at various resolutions maintaining aspect ratios
• Text preprocessing includes normalization of Arabic diacritics
• Synthetic data generation pipeline for augmentation

Citation

If you use this model, cite the original dataset:

@article{heakl2025kitab,
  title={KITAB-Bench: A Comprehensive Multi-Domain Benchmark for Arabic OCR and Document Understanding},
  author={Heakl, Ahmed and Sohail, Abdullah and Ranjan, Mukul and Hossam, Rania and Ahmad, Ghazi and El-Geish, Mohamed and Maher, Omar and Shen, Zhiqiang and Khan, Fahad and Khan, Salman},
  journal={arXiv preprint arXiv:2502.14949},
  year={2025}
}

Related Models

• Base Model: NAMAA-Space/Qari-OCR-0.2.2.1-VL-2B-Instruct
• KITAB-Bench Collection: ahmedheakl/kitab-bench