hunyuanocr-GGUF

HunyuanOCR

HunyuanOCR is a compact vision-language OCR model developed by Tencent, designed for high-quality text recognition, document understanding, layout-aware parsing, and structured information extraction. This repository contains GGUF quantized variants of the model optimized for efficient local inference using llama.cpp.

Unlike conventional OCR systems that focus solely on character recognition, HunyuanOCR combines optical character recognition with visual understanding to interpret document layouts, reading order, tables, forms, and contextual relationships between textual elements. Its lightweight architecture enables efficient deployment while maintaining strong OCR performance across diverse document types.

The quantized formats significantly reduce memory requirements while preserving document understanding and OCR quality, making the model suitable for local document-processing pipelines, enterprise OCR systems, and resource-efficient edge deployments.

---

Model Overview

• Model Name: HunyuanOCR
• Base Model: tencent/HunyuanOCR
• Architecture: Vision-Language Model
• Modalities: Text, Image
• Primary Languages: Multilingual
• Developer: Tencent
• License: Apache 2.0

---

Quantization Formats

This repository provides various GGUF quantized versions of the HunyuanOCR model optimized for efficient local inference using llama.cpp.

IQ3_M

• Size reduction of approx 72.65% (282.96 MB) compared to 16-bit (1.01 GB)
• Aggressive 3-bit quantization optimized for lightweight OCR deployment and rapid document processing
• Suitable for resource-constrained OCR services, edge devices, and large-scale document digitization
• Enables efficient recognition of printed text, scanned documents, invoices, and forms with minimal memory usage
• Fine-grained layout interpretation and challenging document analysis may experience reduced accuracy compared to higher-precision variants

IQ4_NL

• Size reduction of approx 68.50% (325.91 MB) compared to 16-bit (1.01 GB)
• Advanced 4-bit non-linear quantization designed to preserve layout understanding and OCR fidelity
• Better suited for structured document parsing, multilingual OCR, and enterprise document-processing workflows
• Maintains strong consistency across complex page layouts, tables, and mixed-format documents
• May require slightly increased computational overhead during inference

IQ4_XS

• Size reduction of approx 69.64% (313.87 MB) compared to 16-bit (1.01 GB)
• Balanced 4-bit quantization optimized for efficient OCR inference and dependable document understanding
• Provides a practical balance between memory efficiency, recognition quality, and runtime performance
• Suitable for document indexing, intelligent search, archive digitization, and production OCR pipelines
• Maintains stable performance across a broad range of real-world document analysis workloads

---

Training Background (Original Model)

HunyuanOCR is trained with an emphasis on optical character recognition, document parsing, visual-text alignment, and multilingual document understanding across diverse document-centric datasets.

Pretraining

• Large-scale multimodal pretraining using document images and multilingual textual corpora
• Focus on visual-text representation learning, layout understanding, and OCR capability
• Optimized for downstream document intelligence and structured information extraction

Instruction Tuning

• Further refined using OCR-oriented and document understanding datasets
• Enhanced for layout-aware recognition, structured extraction, and document reasoning tasks
• Improved consistency across forms, invoices, reports, tables, and complex document layouts

---

Key Capabilities

• Optical Character Recognition (OCR) Accurately extracts textual content from scanned documents, images, and printed materials.

• Document Understanding Interprets document structure and semantic relationships between textual regions.

• Layout Analysis Recognizes tables, forms, paragraphs, reading order, and page organization.

• Structured Information Extraction Extracts meaningful structured information from business and technical documents.

• Multilingual Processing Supports OCR across multiple languages and document formats.

• Efficient Local Deployment Quantized variants enable practical OCR inference on consumer hardware and edge devices.

---

Usage Example

Using llama.cpp

./llama-mtmd-cli \
  -m SandLogicTechnologies/HunyuanOCR_IQ4_NL.gguf \
  --mmproj mmproj.gguf \
  --image invoice.png \
  -p "Extract all text and preserve the document structure."

---

Recommended Usecases

• Enterprise OCR Digitize business documents with layout-aware text recognition.

• Document Processing Extract structured information from reports, forms, and invoices.

• Archive Modernization Convert scanned archives into searchable digital content.

• Knowledge Management Prepare structured documents for indexing and retrieval systems.

• Document AI Pipelines Power intelligent OCR and document-understanding workflows.

• Research & Evaluation Benchmark OCR, document parsing, and layout-aware vision-language models.

---

Acknowledgments

These quantized models are based on the original work by the Tencent Hunyuan development team.

Special thanks to:

• The Tencent team for developing and releasing the HunyuanOCR model.

• Georgi Gerganov and the llama.cpp open-source community for enabling efficient quantization and inference via the GGUF format.

---

Contact

For questions, feedback, or support, please reach out at [email protected] or visit https://www.sandlogic.com/