trocr-ukrainian-handwritten

TrOCR Fine-tuned on the MultiHTR Ukrainian Dataset (v2.0)

This model is a fine-tuned version of kazars24/trocr-base-handwritten-ru for recognizing Ukrainian handwritten text. It was trained on the datasets used for the MultiHTR Ukrainian Transkribus models (see Tikhonov & Rabus 2024).

Version 2.0 (October 2026) replaces the model published in April 2026. Version 2.0 was retrained on a corrected dataset and reaches a substantially lower character error rate than version 1.0 (4.99% on held-out pages). The earlier model remains available, see Versions.

Model Description

TrOCR (Transformer-based OCR) is a vision-to-text model using a ViT encoder and a causal language model decoder. This version is fine-tuned specifically on Ukrainian handwriting from the 19th and 20th centuries.

• Base model: kazars24/trocr-base-handwritten-ru
• Fine-tuned on: MultiHTR Ukrainian dataset (Achim Rabus, Aleksej Tikhonov et al., University of Freiburg) — the same data used for training the Transkribus models Ukrainian generic handwriting 1 and Ukrainian generic handwritten and typed
• Training data language: Ukrainian only
• Intended use: OCR/HTR of handwritten Ukrainian manuscripts and documents

Training Data

• Sources: Vernadskyi National Library of Ukraine (manuscripts by Taras Shevchenko and Klyment Kvitka); National Museum of the Holodomor-Genocide in Kyiv; GRAC corpus (Lviv Polytechnic University); Prozhito Project; Foundation of the International Memorial Association; for more detailed acknowledgements, see the links above
• Scope: Ukrainian handwritten manuscripts and documents, primarily 19th–20th century
• Training set: 24,705 lines from 773 pages
• Validation set: 970 lines from 28 separate pages that do not occur in training
• Line images: bounding box of each text line (5 px padding), page orientation corrected from EXIF before cropping, resized to 128 px height with aspect ratio preserved; no background normalization, no polygon masking

Performance

Measured on the 970 validation lines with beam search (4 beams), no repeat block and max_length=128; text compared after Unicode NFD normalization and whitespace collapsing. The best checkpoint was selected on the same validation set, so the figure is slightly optimistic.

Training Details

Decoding

The stored generation_config.json decodes with 4 beams, no_repeat_ngram_size=0 and max_length=128, which are the settings used for the figures above. The tokenizer splits Cyrillic into single letters or smaller pieces, so a repeat block suppresses ordinary letter sequences that recur within a line, and a limit of 64 tokens cuts off long lines.

How to Use

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image

processor = TrOCRProcessor.from_pretrained("cyrillic-trocr/trocr-ukrainian-handwritten")
model = VisionEncoderDecoderModel.from_pretrained("cyrillic-trocr/trocr-ukrainian-handwritten")

# The model was trained on line images normalized to 128 px height with aspect
# ratio preserved. Reproduce that before the processor (which otherwise squashes
# directly to 384x384), to match the training distribution.
def resize_to_height(img, height=128):
    w, h = img.size
    return img.resize((max(1, round(w * height / h)), height), Image.LANCZOS)

image = resize_to_height(Image.open("line_image.png").convert("RGB"))
pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values, num_beams=4, no_repeat_ngram_size=0, max_length=128)
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)

Note: Input should be a single text line image, not a full page.

Recommended: Polyscriptor

For real-world use on whole manuscript pages, this model is best run through Polyscriptor, a multi-engine HTR training and comparison tool. Polyscriptor handles automatic line segmentation, full-page batch processing, PAGE XML import/export, and lets you compare this TrOCR model directly against the CRNN-CTC model and other engines on your own material. It is available as a browser UI, a PyQt UI, and a command-line interface.

Versions

To load an earlier version, pass the revision:

model = VisionEncoderDecoderModel.from_pretrained("cyrillic-trocr/trocr-ukrainian-handwritten", revision="v1.0")
processor = TrOCRProcessor.from_pretrained("cyrillic-trocr/trocr-ukrainian-handwritten", revision="v1.0")

Acknowledgements

Funded by the Ministry of Science, Research and the Arts of Baden-Württemberg as part of the digital@bw digitization strategy.

Citation

If you use this model, please cite:

@article{tikhonov_rabus_2024,
  author    = {Tikhonov, Aleksej and Rabus, Achim},
  title     = {Handwritten Text Recognition of Ukrainian Manuscripts in the 21st Century:
               Possibilities, Challenges, and the Future of the First Generic AI-based Model},
  journal   = {Kyiv-Mohyla Humanities Journal},
  volume    = {11},
  year      = {2024},
  pages     = {226--247},
  doi       = {10.18523/2313-4895.11.2024.226-247}
}

MultiHTR project funded by the Ministry of Science, Research and the Arts of Baden-Württemberg (digital@bw).

TrOCR architecture:

@article{li2021trocr,
  title   = {TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models},
  author  = {Li, Minghao and Lv, Tengchao and Chen, Jingye and Cui, Lei and Lu, Yijuan and
             Florencio, Dinei and Zhang, Cha and Li, Zhoujun and Wei, Furu},
  journal = {arXiv preprint arXiv:2109.10282},
  year    = {2021}
}