next-step-omr

Next-Step-OMR

Next-Step-OMR (NSOMR) is a research project on optical music recognition for real scanned piano music: the skewed, faded and ink-bleeding scans found on IMSLP, rather than clean engravings.

This is a project page. No code or weights are released here yet.

What it does

• Input: a scanned page of piano music.
• Output: MusicXML for the whole page, covering pitch, rhythm, voices, chords, ties, key and time signatures, clefs, ottavas, tremolos and dynamics.
• Pipeline: the page is split into systems, and each system is transcribed separately. A post-pass cleans up the notes, and the systems are then joined back into one score.

Model

The current model, AR8, is an image-to-sequence transformer.

• Encoder: a ConvNeXt V2 backbone with fusion blocks and 2D rotary positions. Images are scaled so that the staff-line spacing is constant.
• Decoder: a causal transformer that writes a compact music token language with explicit bars and durations. A deterministic writer turns those tokens into MusicXML.
• Training happens in two stages:
  1. Pretraining on a large corpus of synthetic renders of public symbolic scores (KernScores, PDMX and GrandStaff lineages).
  2. Fine-tuning on real scans: human-verified transcriptions of IMSLP systems, plus OpenScore Lieder scores (CC0) aligned to their IMSLP scans.

Nothing from the IMSLP Piano Scores benchmark below is used in training or tuning.

Results

IMSLP Piano Scores leaderboard (LEGATO benchmark)

The public leaderboard scores full-page MusicXML with OMR-NED (musicdiff). Lower is better.

When it was submitted, NSOMR-AR8-A2-SB ranked second on the board, behind copista-28m (0.1643) and ahead of copista-2m (0.2200).

A2-SB uses the same model and the same predictions as A2. The only change is a writer step that adds secondary beams (16ths, 32nds and so on) to the MusicXML.

OpenScore Lieder scans (held-out pages)

These are 144 held-out pages of OpenScore Lieder songs, paired with their IMSLP scans and scored on the piano part with OMR-NED. Lower is better.

The 95% bootstrap interval for the gap is [0.051, 0.106], and NSOMR is better on 106 of the 144 pages.

This set is in-distribution for NSOMR, which fine-tunes on other Lieder scans, so read it as a domain result, not a generalisation test. The leaderboard above is the out-of-domain number.

Known limitations

• Beams: the model reads primary beams only. Secondary beams are added by a rule-based writer step that works them out from note durations, not from the image. Unusual groupings, such as a secondary beam broken inside a beat, are not reproduced.
• Hairpins: crescendo and diminuendo wedges are not produced.
• Tuplets: compound meters are sometimes read as simple meters with triplets.
• Page assembly: systems are transcribed one at a time. Joining them into a page loses a little accuracy, mostly at system boundaries (courtesy clefs, and ties and slurs that cross a line break).
• Scope: the model is piano only. On songs and chamber scores, only the piano part is transcribed.
• Speed: very long or dense systems are slow to decode.

Status

This is active, results-first research. Code, weights or labelled data may be released later. Nothing here is production-ready. For questions, open a discussion on this repo.

Related

• Project on GitHub: https://github.com/weselyj/nsomr
• LEGATO and the IMSLP Piano Scores dataset: arXiv:2506.19065
• OMR-NED: arXiv:2506.10488