PaddleOCR-VL-1.6-MLX-16bit

PaddleOCR-VL-1.6-MLX-16bit

MLX 16-bit (bfloat16) conversion of PaddlePaddle/PaddleOCR-VL-1.6, a 0.9B-parameter vision-language model for document parsing. Converted for Apple Silicon, running locally with mlx-vlm.

This is the full-precision (unquantized) sibling of OpenGryd/PaddleOCR-VL-1.6-MLX-8bit.

Model details

• Base model: PaddlePaddle/PaddleOCR-VL-1.6 (0.9B, ERNIE-4.5-based)
• Precision: 16-bit bfloat16 (MLX mlx_vlm convert, no quantization), ~1.81 GB on disk
• Architecture: PaddleOCRVLForConditionalGeneration (custom paddleocr_vl model type; trust_remote_code required)
• Context: up to 131,072 tokens
• Vision encoder: 14px patches, 784 px per merged token; resolution cap raised to 4,064,256 px (see below)

Resolution budget

This repo ships a higher vision resolution cap than upstream: max_pixels = 4,064,256 (upstream 1.6 ships 1,003,520; min_pixels stays 112,896). This is deliberate small-text OCR tuning — full pages are no longer downscaled below ~5,000 merged vision tokens. Under mlx-vlm 0.7.x the cap is read from preprocessor_config.json at load time; there is no per-request override, so select lower stages client-side by rendering/downscaling the image before sending it.

Caps are naturally expressed as k×k stages of the 28 px/token grid (28 = patch 14 × merge 2):

Token budget = (12k)² merged tokens; non-square images land slightly under budget (aspect-preserving resize on the factor-28 grid). Measured on the 8-bit sibling on Apple Silicon: full-A4 pages cost ~5× the wall time of the upstream cap (vision attention is global over unmerged patches); sub-cap images are unaffected. To revert to upstream behavior, set max_pixels back to 1003520 in preprocessor_config.json.

Capabilities

• Full-page OCR in Markdown and JSON formats
• Layout analysis, table recognition (structure), formula recognition (LaTeX)
• Chart and seal recognition, text spotting (detection + recognition)

Intended uses

• On-device document parsing on Apple Silicon (M-series)
• Highest-fidelity MLX serving of PaddleOCR-VL-1.6 (no quantization delta vs. the bf16 base)
• Local OCR pipelines where no data may leave the machine
• Served via mlx_vlm.server or the MLX Python API with trust_remote_code=True

Usage

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("OpenGryd/PaddleOCR-VL-1.6-MLX-16bit", trust_remote_code=True)

messages = apply_chat_template(
    processor,
    model.config,
    prompt="OCR:",
    num_images=1,
)
output = generate(model, processor, messages, image="media/document.png", max_tokens=4096)
print(output)

Note: with mlx-vlm 0.7.x, include the image via apply_chat_template(...) (or use the server below) — a bare prompt string without an image placeholder produces an empty completion. Keep prompts exactly OCR:-shaped: free-form suffixes can flip the model into spotting mode (inline <|LOC_n|> coordinates).

To serve OpenAI-compatible endpoints:

python -m mlx_vlm.server --model OpenGryd/PaddleOCR-VL-1.6-MLX-16bit --port 8090 --trust-remote-code

Limitations

• ~2× the memory and disk footprint of the 8-bit quant (1.81 GB vs 1.09 GB); use the 8-bit version where footprint matters more than the last bit of fidelity
• Requires trust_remote_code=True (custom modeling code ships with the repo)
• No text-only inputs: image required

Evaluation

The base model achieves 96.3% on OmniDocBench v1.6, leading on text, formula, and table recognition. See the base model card for full benchmarks.

License

Apache-2.0 — inherited from the base model.

Citation

@misc{paddleocrvl2025,
  title={PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training},
  author={PaddlePaddle Team},
  year={2025},
  howpublished={\url{https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6}},
}