crnn-rukopys

crnn-rukopys

A compact CRNN + CTC line recognizer for handwritten Ukrainian document text, trained from scratch on the Rukopys dataset. At ~6.25M parameters with greedy CTC decode it is CPU-friendly — a lightweight alternative to transformer recognizers.

The training data included handwritten, printed and annotation region crops from Rukopys.

TL;DR

Intended use

Recognizing cropped line/region images from handwritten Ukrainian documents, downstream of a layout detector — where a small, fast, CPU-capable recognizer is preferred over a transformer one.

For stronger recognition quality at ~90× the parameter count, see Hukyl/trocr-large-rukopys.

Architecture

CNN feature extractor → BiLSTM sequence model → linear CTC head. No pretrained backbone — trained end-to-end on Rukopys crops.

• CNN: 6 conv layers (1→64→128→256→256→512→512, 3×3), BatchNorm after the 3rd and 5th conv, MaxPool to height 1 (pools 2,2 · 2,2 · (2,1) · (2,1) so width resolution is preserved for the sequence).
• RNN: single bidirectional LSTM, hidden size 256 (512-d output).
• Head: Linear(512 → 256) → log-softmax → CTC (blank index 0).
• Alphabet: 340 characters + CTC blank (341 classes total), built from the silver+gold training vocabulary. Covers Ukrainian Cyrillic (incl. і ї є ґ ў ѣ), Latin, digits, punctuation, and a long tail of typographic / mathematical symbols (–—«»“”№∑∫≈≤≥… etc.).
• Input: grayscale, height-normalised to 32 px, width capped at 512 px, scaled to [-1, 1].

How to use

Note: resize crops with cv2.INTER_AREA — training used it, and INTER_LINEAR aliases on the heavy downscale to 32 px height.

import cv2
import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

IMG_HEIGHT, IMG_MAX_WIDTH = 32, 512


class CRNN(nn.Module):
    def __init__(self, num_classes: int, rnn_hidden: int = 256) -> None:
        super().__init__()
        self.cnn = nn.Sequential(
            nn.Conv2d(1, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2, 2),
            nn.Conv2d(64, 128, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2, 2),
            nn.Conv2d(128, 256, 3, padding=1), nn.BatchNorm2d(256), nn.ReLU(),
            nn.Conv2d(256, 256, 3, padding=1), nn.ReLU(), nn.MaxPool2d((2, 1)),
            nn.Conv2d(256, 512, 3, padding=1), nn.BatchNorm2d(512), nn.ReLU(),
            nn.Conv2d(512, 512, 3, padding=1), nn.ReLU(), nn.MaxPool2d((2, 1)),
        )
        self.rnn = nn.LSTM(512, rnn_hidden, bidirectional=True, batch_first=False)
        self.fc = nn.Linear(rnn_hidden * 2, num_classes)

    def forward(self, x):
        feat = self.cnn(x)
        _b, _c, h, _w = feat.shape
        feat = feat.squeeze(2) if h == 1 else feat.mean(dim=2)
        feat = feat.permute(2, 0, 1)
        out, _ = self.rnn(feat)
        return torch.log_softmax(self.fc(out), dim=-1)


ckpt = torch.load(hf_hub_download("Hukyl/crnn-rukopys", "best.pt"), weights_only=False)
chars = ckpt["chars"]
model = CRNN(len(chars), ckpt["config"]["rnn_hidden"])
model.load_state_dict(ckpt["model_state"])
model.eval()

gray = cv2.imread("line_crop.jpg", cv2.IMREAD_GRAYSCALE)
new_w = max(4, min(int(gray.shape[1] * IMG_HEIGHT / gray.shape[0]), IMG_MAX_WIDTH))
resized = cv2.resize(gray, (new_w, IMG_HEIGHT), interpolation=cv2.INTER_AREA)
tensor = (torch.from_numpy(resized).float().unsqueeze(0) / 127.5 - 1.0).unsqueeze(0)

with torch.inference_mode():
    log_probs = model(tensor)[:, 0, :]

# greedy CTC decode (collapse repeats, drop blank=0)
idx, prev, out = log_probs.argmax(-1).tolist(), None, []
for i in idx:
    if i != prev and i != 0:
        out.append(i)
    prev = i
print("".join(chars[i] for i in out))

The model expects single-line / single-region crops, not full pages.

Training curriculum

Two-stage silver→gold curriculum, trained from scratch. CTC loss (blank=0, zero_infinity), AdamW, OneCycleLR schedule, online augmentation, best checkpoint selected by lowest validation CER.

1. Silver pretrain (20 ep, early stopping): CTC pretrain from scratch on the Rukopys silver split (auto-labeled crops): 119,346 train / 21,061 val.
2. Gold fine-tune (30 ep): warm-start from the stage-1 checkpoint, with the alphabet grown to cover gold-only characters, on the human-labeled gold train split: 18,948 train / 3,345 val crops, stratified at crop level. The published weights are the best-val-CER epoch (29 of 30).

Hyperparameters (as launched)

Online data augmentation

Online augmentation was applied during training, at working resolution: each crop was first downscaled with INTER_AREA to a 64 px working height (2× the model's 32 px input), augmented, then resized to the final 32 px tensor. Only training crops were augmented (no augmented validation was measured).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.

These simulate scanner/paper variation, ink thinning/bleed, and natural handwriting deformation, widening the range of appearances beyond the raw training crops.

Results

Protocol: full held-out gold validation split filtered to the model's classes (3,345 crops, no subsampling), greedy CTC decode, INTER_AREA resize (training parity). Labels are metric-normalized so CER/WER reflect the scored character set.

Overall

Per class

We also acknowledge that printed/annotation n is quite small, so measuring CER against them is quite noisy.

Files

The checkpoint is a self-contained .pt payload: model weights, the alphabet, the architecture config (rnn_hidden, img_height, img_max_width), and the training-time metrics.

Limitations & biases

• Handwritten Ukrainian archival document material only; expect degradation on other scripts, languages, or modern born-digital text.
• The character set and normalization are tuned to the document-OCR metric — outputs are normalized text, not faithful transcription.
• Greedy CTC decode only — no language model or beam search.
• The train/val split is stratified at crop level, not page level — crops from the same page (and writer) appear on both sides, so the reported val CER is optimistic compared to a page-level protocol.
• annotation and printed classes have small training support and higher CER; formula and table are unsupported.
• Single training run — no across-run variance estimate.

Training data & attribution