parseq-s-rukopys

parseq-s-rukopys

A PARSeq-S (permuted autoregressive sequence) line recognizer for single-line handwritten Ukrainian document text, fine-tuned on the Rukopys dataset.

The training data included handwritten, printed, annotation and table region crops from Rukopys.

TL;DR

Intended use

Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream of a line/region detector or on already-segmented lines.

A companion ViT-B variant fine-tuned with the same recipe is available at Hukyl/parseq-b-rukopys.

How to use

Note: the checkpoint is a plain torch.save archive, not transformers-compatible. The payload is a dict:

import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("Hukyl/parseq-s-rukopys", "best.pt")
payload = torch.load(path, map_location="cpu", weights_only=True)
# keys: "model_state" (state_dict), "charset" (str), "config" (dict),
#       "metrics" (dict), "epoch" (int)
print(payload["config"])
# {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 384,
#  'enc_num_heads': 6, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12,
#  'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1,
#  'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05}

To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation decoder; see baudm/parseq, Apache-2.0) with the embedded config, build a tokenizer over the embedded charset (token order is [E] + the charset string + [B] + [P]; [E] is id 0), then model.load_state_dict(payload["model_state"]).

The input geometry ships in the checkpoint's config: each crop is resized unconditionally to 48×512 RGB (no aspect preservation; INTER_AREA on downscale — an INTER_LINEAR mismatch aliases on the heavy downscale and degrades accuracy) and normalized (x − 0.5) / 0.5. Output confidence is the mean per-step max-softmax over the decoded sequence.

Architecture

PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive Sequence Models" (ECCV 2022, arXiv:2207.06966). Model code is derived from baudm/parseq (Apache-2.0). Geometry and charset are identical to the pretrain stage it is fine-tuned from.

Charset

A frozen 232-character Cyrillic charset (sha256 d1b9161e…3ff976). Token IDs follow string order; specials are [E] (id 0) first and [B], [P] last. Text is not NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. і ї є ґ and the apostrophe), the Russian-only capitals Э Ё and letters ъ ы ё, digits, punctuation, and a small set of Greek/math symbols. Latin lookalikes that are visually identical to Cyrillic letters (a c e i o y; A B C E H K M O P T X) are deliberately excluded — the target output space is Cyrillic-normalized, so those are folded to their Cyrillic counterparts.

Training curriculum

Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels longer than 100 characters before training.

1. Mixed pretrain — Hukyl/parseq-s-cyrillic-handwritten: general Cyrillic handwriting reader trained from scratch on a real + synthetic mix (~0.75M crops/epoch, synthetic:real ≈ 1.5:1). See the pretrain card for the full mix and recipe.
2. Rukopys gold fine-tune (15 ep, LR 5e-5 peak): warm-started from the stage-1 weights, fine-tuned on the human-labeled Rukopys gold data, with page-level validation split. Note: to incorporate as much of the Rukopys gold split as possible, this checkpoint had a near-zero validation set (~5 pages): 22,293 train / 105 val crops (handwritten 21,381 / annotation 519 / printed 288 / table 142, before charset/length filtering). The published weights are the best-val-CER epoch (11 of 15).

Stage-2 hyperparameters (as launched)

Online data augmentation

Online augmentation was applied during training (default profile). Only training crops were augmented, and every train crop was augmented (identity_prob = 0.0), at the 48×512 working resolution (downscaled before augmenting).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.

Results

Protocol: AR greedy + 1 refinement iteration, page-level gold validation split (val crops come from pages the model never saw in training), seed 42.

Why the numbers come from a sibling

Since this checkpoint included almost all of the gold split, measuring CER on the tiny val split would result in meaningless metrics. Instead, we report CER on the sibling model trained with the identical recipe (same warm-start, hyperparameters, augmentation, and seed) on a standard ~185-page held-out split. We suspect that this model is strictly equal or better than the reported numbers.

Per class

We also acknowledge that printed/annotation/table n is quite small, so measuring CER against them is quite noisy. table (multi-line pipe-separated serialization) is not a target of this model; its CER above 1 is expected.

Limitations & biases

• Handwritten Ukrainian archival document material only; expect degradation on other scripts, languages, or modern born-digital text.
• The character set and normalization are tuned to the document-OCR metric — outputs are normalized text, not faithful transcription. Latin lookalikes are folded to their Cyrillic counterparts, so the model never emits the Latin forms.
• printed and annotation have small training support and high CER; table and formula are unsupported.
• Headline metrics are from an identical-recipe sibling, not this exact checkpoint (its own holdout is too small to score).
• Single seed — no across-run variance estimate.

Training data & attribution

License & lineage

Model weights are released under CC BY-SA 4.0 — the most restrictive of the training inputs now that Rukopys is CC BY 4.0 (UkrHandwritten is share-alike, and shares this requirement down to the fine-tuned weights). The PARSeq model code is Apache-2.0 (baudm/parseq). The stage-1 pretrain checkpoint, which did not train on Rukopys, is released under CC BY-SA 4.0.