flova-omr-GGUF

Flova/omr_transformer — GGUF (for CrispEmbed)

GGUF conversions of Flova/omr_transformer, an end-to-end handwritten / whiteboard Optical Music Recognition model that transcribes monophonic "simple notes" staff images into LilyPond note strings (e.g. c'2 a''8 c''8 r4 c'1 …). It is the only permissively-licensed (Apache-2.0) end-to-end handwritten-music OMR model, and complements the printed-music engines (SMT, Polyphonic-TrOMR) already in CrispEmbed.

Architecture

A Donut Vision-Encoder-Decoder (~143M params):

• Encoder — DonutSwin (Swin-Base scale): patch 4, window 10, embed_dim 128, depths [2,2,14,2], heads [4,8,16,32], hidden 1024, input 583×409.
• Decoder — 4-layer pre-norm mBART: d_model 1024, 16 heads, ffn 4096, vocab 75, learned positions (offset +2), scale-embedding (×√1024), GELU. decoder_start/bos = 56 (<s>), eos = 54 (</s>), pad = 55.

Files

The Swin encoder is all-linear, so q8_0 needs no keep-guard: the patch-embed convolution and the relative-position index table are kept at F32 automatically.

Parity

Validated per-stage against the HF reference (CPU): every stage cosine 1.000000 (encoder stages + output, all 4 decoder blocks, logits), 40/40 teacher-forced argmax agreement, and byte-exact greedy decode matching the model card. q8_0 reproduces the exact LilyPond output on all three sample images.

Usage (CrispEmbed)

# auto-downloads flova-q8_0.gguf on first use
crispembed --model flova --ocr score.png
# → c'2 a''8 c''8 r4 c'1 e'8 c'8 c'8 a''8 f'4 a'8 c'8

or with an explicit path:

crispembed -m flova-q8_0.gguf --ocr score.png

CrispEmbed: https://github.com/CrispStrobe/CrispEmbed

License & attribution

Apache-2.0, inherited from the source model.

• Model: Flova/omr_transformer
• Blueprint / project: UHHRobotics22-23/robot_project (marimbabot_vision)

These are format conversions only; all model weights and credit belong to the original authors.

Provenance and EU AI Act Art. 53 note

• Upstream model: Flova/omr_transformer — published by Flova.
• Upstream licence: apache-2.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• What was done here: format conversion and/or quantisation only (GGUF). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.