Warraq-Arabic-HTR-7B

الورّاق — Warraq · Arabic Handwriting → Text

Warraq reads real-world handwritten Arabic — notebook pages, phone photos, forms, cheque amounts — and returns logical-order Arabic Unicode text. It is a 0.5%-of-weights QLoRA adapter on QCRI/Fanar-2-Oryx-IVU, an Arabic-first vision-language model, trained on a deliberately mixed diet of real handwriting so it reads ink instead of reciting a dataset.

What it does on real photos

These are transcripts from actual phone-photo runs (hand-picked demos — measured numbers are in the next section and come only from the benchmark):

It also ships with the practical machinery real documents need: an automatic line splitter for multi-line pages (photo-robust: rulings, cross-line descenders, stanza marks) and grammar-constrained decoding for cheque amounts (snaps a noisy reading to the nearest valid Arabic amount phrase).

Measured performance

On the held-out KHATT v1.0 test split (1038 real handwritten lines), verbatim scoring (diacritics counted), greedy decoding:

Fair-comparison note, read it: Warraq's training data includes the KHATT training split, so its KHATT-test score is an in-domain result. The other models never saw KHATT — they are measured zero-shot under the same prompt, decoding and normalization. The table answers "who reads this handwriting best today", not "whose base model is smartest". Full protocol, all 9 models, distribution stats and charts: evaluation report on GitHub · raw numbers: benchmark.json.

Quickstart

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
from PIL import Image

BASE = "QCRI/Fanar-2-Oryx-IVU"
processor = AutoProcessor.from_pretrained(BASE)
model = AutoModelForImageTextToText.from_pretrained(BASE, device_map="auto")
model = PeftModel.from_pretrained(model, "mabdulaziz499/Warraq-Arabic-HTR-7B")

image = Image.open("line.jpg")   # one handwritten LINE works best
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": "اكتب النص المكتوب بخط اليد في هذه الصورة كما هو تماما."},
]}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:],
                       skip_special_tokens=True))

For multi-line pages, split into lines first (any line segmenter works); for cheque amounts, constrain the output to the Arabic amount grammar — both behaviors are what the demos above used.

<details> <summary><b>Training recipe (click to expand)</b></summary> </details>

Honest limitations

• Ornamental / fully-vocalized calligraphy (Quranic naskh, decorative scripts) is a different domain — Warraq targets everyday handwriting and will under-read calligraphy.
• Historical manuscripts improve over the base but remain hard.
• Very low-resolution crops of fused number-words can garble; use the whole crop plus amount-grammar snapping, as banks do.
• Trained and evaluated on line-level images; page reading relies on a line splitter.

License & credits

Adapter: Apache-2.0 (matching the base model). Base: QCRI/Fanar-2-Oryx-IVU by QCRI. Training data credits: KHATT v1.0 (research terms), OnlineKhatt, Muharaf. If you use Warraq, please also cite KHATT (Mahmoud et al., Pattern Recognition 2014) and the base model.