Downloads · 30 days
42
16% of all-time downloads
qenme/jina-ocr-v1-Q8_0-mtp-gguf
jina-ocr-v1-Q8_0-mtp-gguf is a image-text-to-text model from qenme. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as cc-by-nc-4.0.
I tested jina-ocr-v1 and found it very good at document OCR. Its Multi-Token Prediction (MTP) draft layers should make it fast under speculative decoding, but I could not get MTP working in upstream llama.cpp, so I fo…
Downloads · 30 days
42
16% of all-time downloads
All-time downloads
261
Public
Repo size
4 GB
Likes
2
Public
Click a slice to open those files.
.gguf4 GB · 100%
From the Hugging Face model README
I tested jina-ocr-v1 and found it very good at document OCR. Its Multi-Token Prediction (MTP) draft layers should make it fast under speculative decoding, but I could not get MTP working in upstream llama.cpp, so I forked it and had an LLM fix the MTP path. This repo just provides the resulting GGUF so others can use it.
Fork (required for the fast path): https://github.com/vektorprime/jinaocr_llama.cpp
jina-ocr-v1-Q8_0-mtp.gguf (~3.2 GB) - Q8_0 quant with 3 MTP draft layers keptmmproj-jina-ocr-v1-F16.gguf (~825 MB) - vision projector, unchangedTrunk weights are identical to a plain Q8_0 quant, so the file also loads on any llama.cpp build - the draft heads are simply ignored without the flags below.
llama-server -m jina-ocr-v1-Q8_0-mtp.gguf \
--mmproj mmproj-jina-ocr-v1-F16.gguf \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0.0 \
--jinja -c 32768
Measured ~0.73 draft acceptance on document pages, output identical to non-speculative decoding.
Base model: jinaai/jina-ocr-v1. Quantized with the fork above. License follows the base model (CC-BY-NC-4.0).