OvisOCR2-MLX-BF16

OvisOCR2-MLX-BF16

这是 ATH-MaaS/OvisOCR2 的非官方 MLX BF16 转换版本，面向 Apple Silicon Mac 上的 MLX-VLM 和 oMLX。

This is an unofficial MLX BF16 conversion of ATH-MaaS/OvisOCR2 for document OCR and document parsing on Apple Silicon Macs. It is a format conversion, not a new training run or a fine-tuned checkpoint.

Model summary

Intended use

Use this checkpoint to extract readable content from document images, including:

• printed Chinese and English text;
• headings, paragraphs and lists;
• tables and basic document layout;
• Markdown-oriented document conversion.

The output quality depends on image resolution, blur, contrast, handwriting, document layout and the prompt. This repository does not claim a new accuracy benchmark; users should validate it on their own document types.

Installation

The simplest runtime is mlx-vlm on an Apple Silicon Mac:

python3 -m pip install -U mlx-vlm huggingface_hub

For a clean environment, use a virtual environment instead of installing packages into the system Python:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip mlx-vlm huggingface_hub

MLX requires Apple Silicon. CUDA, ROCm and ordinary x86 CPU inference are not the target runtime for this repository.

Download

Replace YOUR_HF_USERNAME with the account or organization that publishes this repository:

hf download YOUR_HF_USERNAME/OvisOCR2-MLX-BF16 \
  --local-dir ./OvisOCR2-MLX-BF16

The repository should contain the extracted MLX model files at its root. Do not put the model inside another nested directory, and do not upload only the .tar archive.

Quick start with MLX-VLM

The official mlx-vlm command is mlx_vlm.generate:

mlx_vlm.generate \
  --model ./OvisOCR2-MLX-BF16 \
  --image /path/to/document.png \
  --prompt "Extract all readable content from this image in Markdown. Preserve the original reading order, headings, paragraphs, lists, and table structure as much as possible. Return Markdown only; do not add explanations." \
  --max-tokens 4096 \
  --temperature 0.0

For OCR, thinking is normally unnecessary. Do not pass --enable-thinking unless you intentionally want to test a thinking-style prompt.

A shorter prompt can also be used:

Extract all readable content from the image in Markdown. Preserve the original text and layout as much as possible. Return Markdown only.

Use with oMLX

oMLX treats this checkpoint as a Qwen3.5 vision-language model. Copy the model into the directory configured for oMLX, then start the server:

mkdir -p ~/models
hf download YOUR_HF_USERNAME/OvisOCR2-MLX-BF16 \
  --local-dir ~/models/OvisOCR2-MLX-BF16

omlx serve --model-dir ~/models

Open http://localhost:8000/admin/chat, select the model and upload a document image. If an oMLX installation does not identify the model automatically, set its model type to VLM in the Admin panel. Use a prompt like the one above and keep thinking disabled for normal OCR.

The first request can be slower because the model and Metal resources have to be loaded. Subsequent requests are the more useful measure of inference speed. Cold-start time also depends on whether oMLX has evicted the model, the current memory pressure, the storage device and the installed oMLX/MLX version.

Conversion information

The conversion was performed from the upstream Hugging Face checkpoint with mlx-vlm:

mlx_vlm.convert \
  --hf-path ATH-MaaS/OvisOCR2 \
  --mlx-path OvisOCR2-MLX-BF16

This conversion preserves the upstream model architecture and tokenizer/processor assets while changing the weight format to MLX-compatible BF16 tensors. Re-running a conversion with a different mlx-vlm version may produce small metadata differences; the generated config.json and processor files should be kept together with the weights.

Repository contents

The model repository contains the MLX weight file and the configuration, tokenizer and processor files required by mlx-vlm/oMLX. Typical files include:

README.md
config.json
model.safetensors
processor_config.json
preprocessor_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja

The exact auxiliary filenames may vary slightly with the mlx-vlm release. Do not rename or remove files generated by the converter.

Limitations and safety

• This is an unofficial conversion and is not endorsed by the upstream OvisOCR2 authors.
• OCR can be incorrect on low-resolution, skewed, blurred, handwritten or unusual documents.
• Tables and complex multi-column layouts may require prompt tuning or post-processing.
• BF16 uses more memory and storage than the accompanying 4-bit conversion.
• Generated text must be checked before it is used for legal, financial, medical or other high-stakes purposes.
• Speed depends on the Mac model, image token count, output length, memory pressure, cache state and software versions. No universal tokens-per-second guarantee is made.

License and attribution

The upstream model page identifies ATH-MaaS/OvisOCR2 as Apache-2.0. Please review and comply with the upstream license and attribution requirements when redistributing this conversion. This repository is an unofficial format conversion and does not change the upstream license.

Upstream resources:

• OvisOCR2 model card
• MLX-VLM
• oMLX
• Apple MLX

Release note

This repository contains the BF16 MLX conversion named OvisOCR2-MLX-BF16. It should be used together with the model files in this repository, not with the original Transformers/PyTorch weights directly.