olmOCR-2-7B-1025-MLX-4bit

olmOCR-2-7B-1025-MLX-4bit

This is a 4-bit quantized version of allenai/olmOCR-2-7B-1025 optimized for Apple Silicon using MLX.

Model Description

olmOCR-2 is a state-of-the-art OCR (Optical Character Recognition) vision-language model fine-tuned from Qwen2.5-VL-7B-Instruct. This 4-bit quantized version offers the smallest footprint for resource-constrained devices.

Base Model: allenai/olmOCR-2-7B-1025
Quantization: 4-bit using MLX
Model Size: 4.5 GB (down from ~14 GB BF16)
Size Reduction: ~68%

Performance

olmOCR-2 achieves 82.4 points on olmOCR-Bench, representing state-of-the-art performance for real-world OCR of English-language digitized print documents. The model has been additionally fine-tuned using GRPO RL training to boost performance on:

• Math equations
• Tables
• Complex layouts
• Handwriting

Usage

Requirements

pip install mlx-vlm

Basic Usage

from mlx_vlm import load, generate
from PIL import Image

# Load the model
model, processor = load("richardyoung/olmOCR-2-7B-1025-MLX-4bit")

# Load your image
image = Image.open("document.png")

# Extract text
prompt = "Extract all text from this image."
output = generate(model, processor, image, prompt, max_tokens=2048)
print(output)

Command Line

python -m mlx_vlm.generate \
  --model richardyoung/olmOCR-2-7B-1025-MLX-4bit \
  --image document.png \
  --prompt "Extract all text from this image." \
  --max-tokens 2048

Quantization Details

• Method: MLX native quantization
• Bits: 4-bit
• Group Size: Default
• Recommended for: Users with limited RAM or those prioritizing speed/size

Model Variants

System Requirements

• Platform: Apple Silicon (M1/M2/M3/M4)
• RAM: 6+ GB recommended
• OS: macOS 12.0+

Limitations

• Optimized primarily for English-language printed documents
• May have reduced performance on handwritten text compared to printed text
• Requires Apple Silicon hardware for optimal performance
• Some quality degradation compared to higher-bit quantizations (expected with aggressive compression)

Citation

@article{olmocr2,
  title={olmOCR 2: Unit test rewards for document OCR},
  author={Allen Institute for AI},
  year={2025}
}

License

Apache 2.0 (inherited from base model)

Acknowledgements

• Base model by Allen Institute for AI
• Quantized for MLX by richardyoung
• Built with MLX-VLM

---

Generated with Claude Code

---

Built & maintained by Richard Young · DeepNeuro