midf-sanskrit-ocr

MIDF Sanskrit OCR v1.2

The default pipeline combines a Kraken BLLA v1.2 topline segmenter, Kraken polygon tracing, and a PP-OCRv6 v1.0 Sanskrit line recognizer. Both models train on 1,916 reviewed pages and 18,990 lines from dataset v1.1. BLLA v1.2 adds slant and curvature training and runs with tuned inference settings; it detects the exact line count on 228 of 229 test pages, up from 142 with v1.0. On the 220-page M00638 test panel, the complete pipeline reaches 3.45% CER against 3.66% for v1.0 in the same evaluation run. These pages were held out from training; however, the segmenter inherits older weights that had seen M00638 examples, so this is not a fully manuscript-unseen pipeline result.

Paper

The three-page research note, From Scholarly Transcriptions to 3.71% CER, documents the corpus construction, systems screened, final pipeline, and error analysis. Its LaTeX source and bibliography are included for revision and reuse.

Use

With Python 3.12 and uv:

uvx --from huggingface-hub hf download tadad/midf-sanskrit-ocr --revision main --local-dir sanskrit-ocr
uv run sanskrit-ocr/ocr.py page.jpg -o page.txt --json-output page.json

The command runs locally on CPU by default. --device cuda or --device mps selects a supported accelerator. JSON includes detected boundaries, toplines, reading order, recognized text, compact line-confidence summaries, and a reference-free review-risk assessment. The runner applies EXIF orientation and passes the full image to both models. BLLA runs at a 1,200-pixel input height and drops toplines shorter than 20% of the page's median topline, which removes single marginal characters and stray fragments (segmentation in pipeline.json; the JSON output reports how many were dropped). Kraken normalizes the page to 1,000 pixels high for polygon tracing (scale=[1000, 0], topline=true) and returns polygons in original-image coordinates. PP-OCRv6 receives crops from the full-resolution page and normalizes them to its saved 96-pixel input height.

Reference-free review triage

JSON output includes review.risk_score (0–100), quality_confidence, the inverse review_priority, and human-readable reasons. The score combines PP-OCRv6 character confidence with observable failure signals such as tiny line fragments, unusually thin polygons, polygonization failures, sparse or unexpected output, repetition, and text-bearing pages on which no lines were found. Each line also includes a compact confidence summary.

This is a review-ranking heuristic, not a probability that the text is correct. In a targeted nine-page smoke panel, both pages below 1.4% known CER were high-confidence, the two catastrophic pages above 36% CER were high-priority, and every sampled page above 10% CER was at least medium-priority. See evaluation/review-risk-sample.json.

For archive-scale inference, sort pages by review_priority and risk_score, send the highest-risk pages to human review first, and audit a random sample of high-confidence pages for false negatives. Preserve the component signals and reasons so corrected pages can support later threshold calibration without repeating OCR.

To evaluate the pipeline on the dataset's nine-page legacy evaluation panel:

uv run sanskrit-ocr/evaluation/evaluate.py \
  /path/to/midf-egangotri-sanskrit/data/segmentation/evaluation/v0.1 \
  --output pipeline-evaluation

Complete pipeline evaluation

Page CER compares concatenated page text after NFC normalization and removal of Unicode whitespace, pooled over all reference characters. v1.0 and v1.2 below come from the same evaluation run on the 229-page test split:

The adaptation panel is nine pages, so its change is within page-to-page noise but is reported as measured. Segmentation gains are largest on pages that v1.0 fragmented badly, most of which have no reference transcription; see the segmentation notes.

The published v1.0 evaluation, from an earlier run of the same references, recorded:

The M00638 panel contains a single manuscript style and does not establish unrestricted Sanskrit manuscript OCR performance.

On those same 220 pages, Gemini 3.1 Pro scores 9.86% CER with the same references and normalization. The v1.0 pipeline has 64% fewer character errors on this panel. The segmenter's M00638 training exposure still limits the interpretation of this comparison. See the matched-page Gemini evaluation for predictions, settings, and cost.

Aggregate evaluation metrics provide the evaluation record.

Recognition evaluation

These scores evaluate the recognizer on reviewed line geometry. They are diagnostic line-crop metrics, not complete-pipeline page CER.

Line-level evaluation penalizes text assigned to an adjacent crop. Complete page evaluation concatenates the lines and measures the OCR output users receive. Recognition details describe the checkpoint and metrics.

Training and selection

• BLLA v1.2 continues from v1.0 with learning rate 5e-5, a slant/curve augmenter (rotation up to 6 degrees, row-bending warps), a 12-pixel topline target, and slanted or curved training pages counted twice. A 60-page validation panel selects epoch 4 in one-based numbering; the run stops at its 3.5-hour training limit after six epochs. BLLA v1.0 continued from v0.3 in the same way without the slant/curve changes.
• PP-OCRv6 v1.0 continues from the strongest pre-v1.1 checkpoint with learning rate 1e-4, a cosine schedule, 200 warmup steps, batch size 8, augmentation, NFC targets, and no bidi reordering. The 9,514-line validation set selects epoch 8 in one-based numbering. The job evaluates the selected checkpoint after its 2.5-hour training budget.
• Both jobs use Kraken 7.1.1 and dataset revision fa66bc54. M00624, M00628, and M00668 supply validation. M00638 supplies the main test panel, while exact M00626 and M00664 test pages remain outside v1.1 training. The recognizer is manuscript-unseen on M00638 because its clean v0.9 parent began from the original PP-OCRv6 pretrain and excluded M00638. The segmenter resumes v0.3, whose earlier training included M00638 examples.

Segmentation provenance and recognition provenance record the full settings, checksums, jobs, and raw logs.

Limitations

The complete pipeline detects the exact reference line count on 228 of 229 test pages, but the test panel is dominated by one clean manuscript. On corpus collections outside the training manuscripts, BLLA v1.2 still fragments some pages: very large script, dense folios with interlinear insertions, curled page edges, ruled frames and decoration, and tables. Margins, crossed-out writing, diagrams, water damage, and dense layouts require review. Body-text transcriptions do not consistently represent every visible mark.

The main evaluation covers M00638, but the complete pipeline is not fully manuscript-unseen because of the segmenter's inherited training history. Additional clean manuscript-held-out panels are required to estimate performance across other scripts, hands, inks, and layouts.

Repository layout

ocr.py runs the released pipeline using the weights declared in pipeline.json. Segmentation and recognition contain the current model weights, provenance, and component metrics. Training contains the final training runner, and evaluation contains the end-to-end evaluation runner and aggregate metrics. Historical checkpoints remain available through Git history and release tags.

Attribution and terms

The PP-OCRv6 base weights are by Benjamin Kiessling / Inria and released under Apache-2.0; see upstream attribution and the included license. This fine-tune modifies those weights for Sanskrit recognition. Kraken supplies segmentation, recognition, and training software under its upstream terms. MIDF/eGangotri images and scholarly transcriptions retain the provenance and usage terms in the dataset card, which lists its license as other. This release grants no additional source-data rights and is not an official MIDF, Inria, PaddlePaddle, Google, or Qwen release.