GOT-OCR2_0-bf16

mlx-community/GOT-OCR2_0-bf16

bf16 MLX conversion of stepfun-ai/GOT-OCR2_0, the 560M-parameter OCR-2.0 model. Converted with mlx-vlm 0.6.14 (mlx 0.32.0) for inference on Apple Silicon.

Fidelity was measured against the unquantized bf16 source and against the reference PyTorch implementation, not assumed. Every number below comes from a run recorded in this card.

Requires mlx-vlm with GOT-OCR 2.0 support, which is currently open as Blaizzy/mlx-vlm#1908. The PR is not merged yet, so a released mlx-vlm cannot load this repo.

python -m mlx_vlm generate \
    --model mlx-community/GOT-OCR2_0-bf16 \
    --image document.png \
    --prompt "OCR: " \
    --max-tokens 1024

GOT is not a chat model. It takes two instructions: OCR: for plain text and OCR with format: for structured output (tables, formulas, sheet music). Any other prompt is out of distribution.

Conversion

No quantization. This is the reference the two quantized variants were measured against, and the one to use when fidelity matters more than memory.

Against the reference PyTorch implementation

One forward pass, same image and same prompt (the MPT conversation the model was trained with), comparing the final-position logits over all 151,860 classes.

The second row is the one that matters: the source weights are bf16, so running the MLX port in fp32 loses nothing, and the residual is porting error alone. The first row's larger figure is bf16 rounding, not a defect.

Task level, against ground truth

Six documents rendered locally with exactly known text (invoice, lab report, shipping label, receipt, spec table, rotated receipt), transcribed with OCR: and scored directly. This sidesteps using bf16 as the reference at all.

field counts required strings present exactly, content ignores markup and whitespace, numeric counts ground-truth numbers recovered.

The 8-bit output is byte-identical to bf16 on all six documents, down to the same fields missed.

Two caveats a reader should have:

• The 4-bit field score is higher than bf16 and that is not an improvement. The denominator is 38 strings, so one string is 0.026. 4-bit happens to get the invoice fully right and then loses 12.00 on the receipt. content and numeric, which are less format-sensitive, both move the other way. Treat the three variants as separated by one or two strings, not by field.
• shipping_label loses the same three fields in every variant, bf16 included (1Z-994-AX-77310582, Halberd Components, 3011AB). That is a limit of the source model on that document, not a quantization effect.

Throughput measured on an M-series Mac during the same run, single image, generation only.

What was not measured

• Only the plain 1024x1024 single-crop path. GOT's fine-grained mode (region by box or by colour), its multi-crop path for dense pages, and its multi-page mode are not exercised by anything above.
• OCR with format: was not scored. The accuracy table uses OCR: only. Format mode was smoke-tested and runs, but no ground-truth scoring was done for tables, formulas or sheet music.
• No perplexity, top-1 agreement or KL. GOT only emits transcriptions, so generic passages give meaningless perplexity. The ground-truth OCR scoring above replaces that layer rather than supplementing it.
• Six synthetic documents, one language. They are rendered, not photographed, and English only. Real scans, handwriting and the multilingual claims of the source are untested here.
• No accuracy measurement at long context. Everything ran well under the 32k window.

Related

• mlx-community/GOT-OCR2_0-bf16
• mlx-community/GOT-OCR2_0-8bit
• mlx-community/GOT-OCR2_0-4bit

Source model: stepfun-ai/GOT-OCR2_0 (paper).