chess-ocr-bilstm

Chess OCR BiLSTM

A BiLSTM neural network for recognizing handwritten chess moves from scoresheet images.

Model Description

This model recognizes handwritten chess moves in Standard Algebraic Notation (SAN) from cropped move-box images extracted from chess scoresheets.

• Architecture: CNN frontend + Bidirectional LSTM + CTC loss
• Input: 64×256 grayscale image, normalized to [-1, 1]
• Output: Chess move string (e.g., "e4", "Nf3", "O-O", "Qxd7+")
• Character set: 28 classes (blank + K, Q, R, B, N, a-h, 1-8, x, +, #, =, O, -)

Model Variants

Usage (Browser - ONNX Runtime Web)

import * as ort from 'onnxruntime-web';

// Load model from HuggingFace
const modelUrl = 'https://huggingface.co/GerhardTrippen/chess-ocr-bilstm/resolve/main/chess_ocr.onnx';
const session = await ort.InferenceSession.create(modelUrl);

// Preprocess image to 64x256 grayscale, normalize to [-1, 1]
const inputTensor = new ort.Tensor('float32', imageData, [1, 1, 64, 256]);

// Run inference
const results = await session.run({ input: inputTensor });
const output = results.output.data;

// Decode with CTC (greedy or beam search)
const move = ctcDecode(output, charset);

Usage (Python/PyTorch)

import torch
from PIL import Image
import numpy as np

# Load model
checkpoint = torch.load('chess_ocr_v47_complete.pth', map_location='cpu')
model = ChessOCRModel()  # See training notebook for architecture
model.load_state_dict(checkpoint['model_state_dict'])
model.eval()

idx_to_char = checkpoint['idx_to_char']

# Preprocess: resize to 256x64, grayscale, normalize
image = Image.open('move_cell.png').convert('L')
image = image.resize((256, 64))
img_array = np.array(image, dtype=np.float32)
img_array = (img_array / 255.0 - 0.5) / 0.5  # Normalize to [-1, 1]
tensor = torch.FloatTensor(img_array).unsqueeze(0).unsqueeze(0)

# Inference
with torch.no_grad():
    output = model(tensor)
    
# CTC greedy decode
predictions = output.argmax(dim=2).squeeze()
move = ''.join([idx_to_char[idx.item()] for idx in predictions if idx.item() != 0])

Training

Trained on the HCS (Handwritten Chess Scoresheet) dataset.

See BiLSTM_v7.ipynb for the complete training code.

Training configuration:

• Epochs: 50
• Batch size: 32
• Learning rate: 0.0005
• Optimizer: Adam
• Augmentation: 10:1 ratio (rotation ±5°, horizontal scale, shear, vertical shift, brightness/contrast)

Key differences from original Eicher et al. approach:

• Trained from scratch (no IAM pretraining)
• Modified preprocessing: no horizontal line removal, center-padded alignment
• Achieves comparable results: 82.9% MRA vs 82.5% MRA (Eicher 2021)

Acknowledgments & Citations

This model implements the architecture described in:

Eicher, O., Farmer, D., Li, Y., & Majid, N. (2021). "Handwritten Chess Scoresheet Recognition Using a Convolutional BiLSTM Network." ICDAR 2021 Workshops, pp. 245-259.
DOI: 10.1007/978-3-030-86198-8_18

Majid, N., Eicher, O. (2022). "Digitization of Handwritten Chess Scoresheets with a BiLSTM Network." J. Imaging 2022, 8, 31.
DOI: 10.3390/jimaging8020031

Dataset:

• Original HCS dataset by Eicher, Farmer, Li, Majid
• HuggingFace hosting by Benjamin Kost

License

MIT License - Free for personal and commercial use.

Author

Trained and uploaded by Gerhard Trippen (2025).

Citation

@misc{trippen2025chessocr,
  author = {Trippen, Gerhard},
  title = {Chess OCR BiLSTM: Handwritten Chess Scoresheet Recognition},
  year = {2025},
  publisher = {HuggingFace},
  url = {https://huggingface.co/GerhardTrippen/chess-ocr-bilstm}
}