trocr-large-uk-handwritten

trocr-large-uk-handwritten

A TrOCR-Large (VisionEncoderDecoderModel, ~558M params) recognizer for line-level handwritten Ukrainian text, trained on a mix of real and synthetic Cyrillic handwriting corpora.

This checkpoint is a general-purpose handwriting reader, meant as a starting point for downstream HTR fine-tunes. A real-data-only refinement of this checkpoint is published separately as Hukyl/trocr-large-uk-handwritten-real.

TL;DR

Intended use

Recognizing single cropped lines of handwritten Ukrainian text, downstream of a line/region detector or on pre-segmented lines. Useful directly as a general handwriting reader, or as an init for fine-tuning on a target domain.

How to use

Note: load the processor from this repo, not from microsoft/trocr-large-handwritten — the tokenizer here is an extended Cyrillic vocabulary (50336 vs stock 50265).

from PIL import Image
from transformers import VisionEncoderDecoderModel, TrOCRProcessor

repo = "Hukyl/trocr-large-uk-handwritten"
processor = TrOCRProcessor.from_pretrained(repo)          # load from THIS repo
model = VisionEncoderDecoderModel.from_pretrained(repo).eval()

crop = Image.open("line_crop.png").convert("RGB")          # one cropped text line
pixel_values = processor(images=crop, return_tensors="pt").pixel_values
generated = model.generate(pixel_values, max_new_tokens=256, num_beams=1)  # greedy
text = processor.batch_decode(generated, skip_special_tokens=True)[0]
print(text)

Lineage & training curriculum

Full fine-tune, optimizer: AdamW via the 🤗 Seq2SeqTrainer, cosine LR decay, max_target_length 256, online augmentation, seed 42, single L40S.

1. Base — Kansallisarkisto/cyrillic-htr-model (Apache-2.0): microsoft/trocr-large-handwritten fine-tuned on ~30k historical Cyrillic rows by the Finnish National Archives; extended tokenizer (vocab 50336).
2. Mixed pretrain (6 ep, LR 4e-5, warmup 0.1): UkrHandwritten (oversampled ×3) + Cyrillic Handwriting Dataset + pumb-ai/synthetic-cyrillic-large capped at 555k, for a synthetic:real ratio ≈ 3:1 (≈579k crops per epoch). The published weights are the best-val-CER epoch (6 of 6).

Hyperparameters (as launched)

Training ran 115,662 steps in ~15.9h.

Online data augmentation

Online augmentation was applied during training. Only training crops were augmented (no augmented validation was measured).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. All training crops are handwritten lines, so a single handwriting-oriented transform set was used.

These simulate scanner/paper variation, ink thinning/bleed, and natural handwriting deformation, widening the range of appearances beyond the raw training crops.

Results

In-mix held-out validation

Held-out 0.3% of the training mix (~2k lines drawn from UkrHandwritten + Cyrillic Handwriting + the synthetic corpus, same distribution as training). Greedy decode.

This split mirrors the 3:1 synthetic:real training ratio, so the absolute numbers are not comparable to real-document CER — they measure fit to the pretraining distribution.

Out-of-domain transfer probe (zero-shot)

The same checkpoint evaluated zero-shot on a held-out page-level split of the Rukopys dataset (handwritten Ukrainian document pages). 1000-sample draw, seed 42, greedy decode, max_new_tokens=512.

The gap between in-mix (0.0173) and zero-shot documents (0.1925) is the expected domain shift from clean line corpora to document crops; a fine-tune on in-domain document data closes most of it. We also acknowledge that printed/table/annotation n is quite small, so measuring CER against them is quite noisy.

Limitations & biases

• General, not specialized: expect higher error on noisy archival scans until fine-tuned (see the zero-shot transfer probe).
• A large share of the training mix is synthetic; for a real-data-only consolidation see Hukyl/trocr-large-uk-handwritten-real.
• Single seed and validation split — no across-run variance estimate.

Training data & attribution

Attribution is mandatory for the UkrHandwritten dataset (CC BY-SA 4.0) — please keep this credit if you redistribute or build on this model.