Qwen3.5-ocr-jp-2b

Qwen3.5-OCR-JP-2B

Qwen3.5-OCR-JP-2B is a Japanese/English Vision-Language OCR model built on top of Qwen3.5-2B. Output schema is compatible with Chandra OCR 2 (datalab-to/chandra) — HTML layout blocks with bounding boxes and labels.

Focus

Training data emphasizes the following Japanese document features:

• Ruby annotations — emitted as HTML5 ruby markup, e.g. <ruby>漢字<rt>かんじ</rt></ruby>
• Japanese handwriting, vertical writing

Quickstart

vLLM (recommended)

import base64, io
from PIL import Image
from vllm import LLM, SamplingParams

PROMPT = "OCR this image as HTML layout blocks with bbox and label."

llm = LLM(
    model="ebinan92/Qwen3.5-ocr-jp-2b",
    dtype="bfloat16",
    max_model_len=12288,
    limit_mm_per_prompt={"image": 1},
    trust_remote_code=True,
)
sampling = SamplingParams(temperature=0.0, top_p=0.1, max_tokens=8000)

image = Image.open("page.png").convert("RGB")
buf = io.BytesIO()
image.save(buf, format="PNG")
b64 = base64.b64encode(buf.getvalue()).decode()

messages = [{
    "role": "user",
    "content": [
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
        {"type": "text", "text": PROMPT},
    ],
}]
print(llm.chat(messages, sampling_params=sampling)[0].outputs[0].text)

Requires vllm>=0.19.1 and transformers>=5.5.1.

transformers

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

PROMPT = "OCR this image as HTML layout blocks with bbox and label."

ckpt = "ebinan92/Qwen3.5-ocr-jp-2b"
processor = AutoProcessor.from_pretrained(ckpt)
model = AutoModelForImageTextToText.from_pretrained(
    ckpt, dtype=torch.bfloat16, device_map="auto"
)

image = Image.open("page.png").convert("RGB")
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": PROMPT},
    ],
}]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

out = model.generate(**inputs, max_new_tokens=8000, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])

Benchmarks

sarashina2.2-ocr's olmOCR-bench overall is omitted because its HF card does not report the baseline row.

^※ VJRODa is evaluated on 92 / 100 samples (8 PDFs are NDL WARP-restricted and unavailable).
^† olmOCR-bench score for chandra-ocr-2 is taken from the official HF card.

<details> <summary>olmOCR-bench JSONL breakdown</summary> </details>

Limitations

• Works only with the single fixed prompt above. It is not tuned for other tasks or free-form instructions.
• Trained primarily on Japanese and English. Coverage of other languages (Chinese, Korean, etc.) is incidental.

License

Apache 2.0.

This model is derived from Qwen3.5-2B, trained on independently constructed datasets. No outputs or weights from datalab-to/chandra-ocr-2 (or any other Chandra release) were used.

Acknowledgements

• Qwen3.5-2B — base model (Apache 2.0)
• Chandra — format reference