wayu-paxa-ocr-zero-gguf

Wayu-Paxa-OCR-Zero — GGUF

llama.cpp builds of wayu-ai/wayu-paxa-ocr-zero, a 0.9B Thai document recognizer trained without a single OCR label from a real Thai document. These are for reading pages on a CPU. The safetensors weights, the results and the full model card live in the parent repository.

Every build needs two files: a language model and the vision projector. The projector is never quantized — it is also where a CPU spends most of its time.

Use

This is a region recognizer, not a page reader: it is trained on crops and has never seen a whole page. A page takes two models — PP-DocLayoutV3 finds the regions and orders them, this one transcribes each — which the stock PaddleOCRVL pipeline does for you.

hf download wayu-ai/wayu-paxa-ocr-zero-gguf --local-dir gguf/

llama-server -m gguf/Wayu-Paxa-OCR-Zero-Q4_K_M.gguf \
    --mmproj gguf/mmproj-Wayu-Paxa-OCR-Zero-F16.gguf \
    --alias PaddleOCR-VL-1.6-0.9B --jinja \
    -c 8192 -t 10 -np 4 --port 8080

from paddleocr import PaddleOCRVL

pipeline = PaddleOCRVL(
    pipeline_version="v1.6",
    vl_rec_backend="llama-cpp-server",
    vl_rec_server_url="http://127.0.0.1:8080/v1",
    vl_rec_max_concurrency=4,   # match `-np`; llama.cpp queues the rest
    device="cpu",               # PP-DocLayoutV3 on CPU too
)
result = pipeline.predict(["page.png"], use_queues=False)[0]
print(result.markdown["markdown_texts"])

Three things are load-bearing. --alias PaddleOCR-VL-1.6-0.9B: the pipeline dials the recognizer by that exact name. --jinja: the chat template is embedded in the GGUF and llama-server ignores it otherwise, which makes the model answer as if there were no image. -t: physical cores, not threads — SMT hurts here.

A ready-made driver script and the full tutorial are in wayu-research/wayu-ocr-inference.

What CPU costs you

Measured on ten cores of an i9-10900X reading dense Thai tax invoices: about a minute per page at Q4_K_M, against a second and a half on an RTX 3090. llama-server peaked at 1.74 GiB resident with -c 8192 -np 4; PP-DocLayoutV3 runs beside it in its own process.

Q4_K_M is the fast build, not the accurate one. It holds up on clean born-digital pages and gives way on the long tail — dot-matrix carbon copies, tables, and the other hard pages the model was already unsure of. When the transcript has to be right, read those pages at full precision: F16 here, or the safetensors build on a GPU.

Note that the llama.cpp path cannot clamp image tokens (min_pixels / max_pixels are vLLM-only), so every crop, however small, pays the model's minimum image-token count.

Provenance

Converted from the released safetensors with stock llama.cpp — the paddleocr architecture has been native since #18825, so no patched build is needed:

python convert_hf_to_gguf.py wayu-paxa-ocr-zero/ --outfile Wayu-Paxa-OCR-Zero-F16.gguf --outtype f16
python convert_hf_to_gguf.py wayu-paxa-ocr-zero/ --outfile mmproj-Wayu-Paxa-OCR-Zero-F16.gguf --outtype f16 --mmproj
llama-quantize Wayu-Paxa-OCR-Zero-F16.gguf Wayu-Paxa-OCR-Zero-Q4_K_M.gguf Q4_K_M

Apache-2.0, as the parent model and its base PaddleOCR-VL-1.6. Cite the paper via the parent repository.

Acceptable use. By downloading or using this model you agree to the Wayu Research Acceptable Use terms.