ocr-math-captcha

OCR Match Captcha

A TrOCR model fine-tuned to recognize short mathematical captcha expressions such as 26+7=? and 45-6=?.

<table border="0" cellspacing="24" cellpadding="12"> <tbody> <tr> <td style="border:none;padding:18px;"><img src="assets/captcha-sample-2.png" alt="Handwritten captcha" width="290" height="90"></td> <td style="border:none;padding:18px;"><img src="assets/captcha-sample-3.png" alt="Clean captcha" width="290" height="90"></td> </tr> <tr> <td style="border:none;padding:18px;"><img src="assets/captcha-sample-4.png" alt="Distorted captcha" width="290" height="90"></td> <td style="border:none;padding:18px;"><img src="assets/captcha-sample-1.png" alt="Math captcha" width="290" height="90"></td> </tr> </tbody> </table>

Model details

• Architecture: Vision Encoder-Decoder / TrOCR
• Base model: microsoft/trocr-base-printed
• Task: Image-to-text OCR
• Target geometry: 130x30 RGB images
• Output: Mathematical expressions without spaces

Training configuration

Usage

from PIL import Image
from transformers import TrOCRProcessor, VisionEncoderDecoderModel

repo_id = "arkhabbazan/ocr-match-captcha"
processor = TrOCRProcessor.from_pretrained(repo_id, token=True)
model = VisionEncoderDecoderModel.from_pretrained(repo_id, token=True)

image = Image.open("captcha.jpeg").convert("RGB")
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values, num_beams=4, max_length=32)
prediction = processor.batch_decode(
    generated_ids, skip_special_tokens=True
)[0]
print(prediction.replace(" ", ""))

Limitations

• Training data is primarily synthetic.
• Unseen fonts and layouts may reduce accuracy.
• The model is intended for short expressions, not document OCR.
• Usage must be authorized and compliant with applicable policies.