typhoon-ocr1.5-2b-GGUF

typhoon-ocr1.5-2b GGUF

GGUF quantization of typhoon-ai/typhoon-ocr1.5-2b, a Thai/English document-OCR vision-language model built on Qwen3-VL-2B-Instruct.

This is a vision-language model: you need both the model file and the mmproj (multimodal projector) file. The model file alone will load, but it will not see images.

Files

The Q4_K_M weights were quantized with an importance matrix (imatrix) computed from a calibration set, which recovers some of the quality lost at 4 bits compared to a plain Q4_K_M of the same size.

Usage

llama-server (OpenAI-compatible API)

llama-server \
  -m typhoon-ocr1.5-2b-Q4_K_M-imat.gguf \
  --mmproj typhoon-ocr1.5-2b-mmproj-Q8_0.gguf \
  -c 8192 --host 0.0.0.0 --port 8080

Then post an image to /v1/chat/completions the usual way:

curl http://localhost:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
        {"type": "text", "text": "Extract all text from this document as Markdown."}
      ]
    }]
  }'

llama-mtmd-cli (one-shot)

llama-mtmd-cli \
  -m typhoon-ocr1.5-2b-Q4_K_M-imat.gguf \
  --mmproj typhoon-ocr1.5-2b-mmproj-Q8_0.gguf \
  --image page.png \
  -p "Extract all text from this document as Markdown."

Use a recent llama.cpp build — Qwen3-VL support landed relatively late.

License

Apache 2.0, inherited from the base model. See the base model card for the model's intended use and limitations.