Llama-xReadable-11B-zhTW-OCR-v2

Llama-xReadable-11B-zhTW-OCR-v2

Built with Llama

云碩科技 · xCloudinfo　·　系列：Meta Llama 3.2 Vision · 云碩繁體中文OCR

以 meta-llama/Llama-3.2-11B-Vision-Instruct 為基底，由云碩科技在自有AI算力資源池上微調的繁體中文（台灣）OCR模型。專注台灣場景的文字辨識——公文、票據、文件與各類筆記的中英混寫內容，可地端部署、資料不外流。xReadable：讓文件被讀懂。

已有更新版本

新導入請使用 Llama-xReadable-11B-zhTW-OCR-v3 —— 字型多樣性由5個設計家族擴充至16個，印刷辨識在各項量測上全面優於本版。本頁保留供既有使用者參考。

v2相對v1的變化：真實手寫訓練樣本大幅擴充（含首次納入純英文樣本），長篇手寫辨識大幅改善；短篇手寫與v1相當；印刷／掃描辨識維持生產可用水準。定位：印刷／掃描文件辨識為生產可用主力；手寫辨識持續迭代中。

能力與量測（真實資料、逐字CER，越低越好）

評測皆以真實拍攝／掃描或渲染文件影像計算字元錯誤率（CER）。

未見字型的實測表現

上表0.73% 係在內部測試集上量測，該測試集的渲染字型與訓練資料同源。為量測真實泛化能力，我們另以訓練從未使用過的字型驗證：

本模型對未見過的字型泛化良好：未見宋體0.00%、未見黑體2.78%。

勘誤（2026-07-23）：本卡片先前版本記載「未見宋體25~28%」並據此描述字型泛化不足。該數字係測試影像製作錯誤所致，並非模型表現：Songti.ttc 的第0個字面為簡體字面（Songti SC Black），缺少「碩體學識測試義區號電話機驗證準確」等繁體字；PIL ImageFont.truetype(path, size) 預設取index=0，缺字被靜默渲染為空白（非豆腐字方框），模型正確辨識了影像上實際存在的文字，是比對用的ground truth有誤。改用同一檔案中的Songti TC Regular（index=7）重製測試圖後，實測為 0.00%。特此更正。

提醒自行評測者：渲染測試影像後請逐字確認非空白再用於量測，否則會把字型缺字誤判為模型缺陷。

更強的版本：字型多樣性已於 v3 擴充至16個設計家族（另保留8家族全程不進訓練供泛化量測），未見黑體由2.78% 進步至1.39%。新導入建議直接使用v3。

誠實說明：手寫短篇的64~66% 是「影像難度地板」，非模型天花板

我們以完全不同架構的開源OCR（NVIDIA nemotron-ocr-v2，偵測＋辨識管線、非語言模型）在同一組真實手寫短篇上交叉驗證，逐張CER 與本模型幾乎完全一致（例：某淡鉛筆掃描頁兩者皆約88%、某清晰頁兩者皆約45%）。兩種原理迥異的系統得到相同的逐張結果，代表該數字主要由影像本身的可辨識度決定（淡鉛筆、潦草、低對比＝物理極限），而非單一模型的弱點。清晰書寫的短篇手寫，實測可達40% 上下；影像品質是關鍵變因。

用途與限制

• 適用：台灣繁中印刷／掃描文件的地端OCR、文件數位化、票據與筆記辨識前處理；長篇手寫筆記轉錄（v2已明顯改善）。
• 輸出正規化：本模型輸出統一為繁體中文（訓練標註採繁體正規化）；輸入含簡寫字時亦會輸出對應繁體。
• 限制：
  • 字型多樣性：本版合成訓練語料涵蓋7個字型；未見字型實測0.00%～2.78%。v3 已擴充至16個設計家族，表現更佳。
  • 手寫辨識仍在迭代；潦草、淡筆、低對比字跡可能出錯，重要用途請務必人工複核。
  • 純英文段落已納入訓練但樣本量仍有限。
  • 表格／複雜版面建議先切行／切格再辨識。
  • 輸出僅供辨識參考，不宜作為法律、醫療、財務等關鍵決策的唯一依據。

格式與相容性

本repo提供 bf16 safetensors（transformers／vLLM原生格式），未提供GGUF：Llama 3.2 Vision的 mllama 架構llama.cpp官方尚未支援轉換／載入，故llama.cpp／LM Studio／Ollama目前無法運行本模型；transformers 與 vLLM 可原生載入。上游支援後將補上官方相容GGUF。

硬體需求

bf16權重約21GB，推論建議單張 ≥ 24GB VRAM的GPU（RTX 3090/4090、A10、L4、A100等），或以 device_map="auto" 多卡分攤；記憶體不足可用 bitsandbytes 做int8/int4量化。

用法（transformers）

from transformers import MllamaForConditionalGeneration, AutoProcessor
from PIL import Image
import torch

model_id = "xCloudinfo/Llama-xReadable-11B-zhTW-OCR-v2"
model = MllamaForConditionalGeneration.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open("your_document.jpg").convert("RGB")
prompt = "請辨識圖片中的所有文字，逐字完整輸出，只輸出文字本身。"
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

用法（vLLM）

vllm serve xCloudinfo/Llama-xReadable-11B-zhTW-OCR-v2 --dtype bfloat16 --max-model-len 4096

授權與來源聲明（Llama 3.2 Community License）

本模型為Meta Llama 3.2的衍生模型，散布須遵守 Llama 3.2 Community License Agreement：

• Built with Llama。 本模型以Meta Llama 3.2建構。
• 基底模型：meta-llama/Llama-3.2-11B-Vision-Instruct（Meta Platforms, Inc.）。
• 模型命名：依授權要求，衍生模型名稱以「Llama」開頭。
• 授權文本：本散布隨附Llama 3.2 Community License Agreement副本（見本repo LICENSE）；須提供該授權副本並保留本聲明。
• 可接受使用政策：使用須遵守Meta Acceptable Use Policy。
• 歐盟（EU）限制：Llama 3.2授權規定，設立地或主要營業地在歐盟的個人或公司，不被授予使用Llama 3.2多模態模型所含之多模態能力的授權。
• 大規模服務門檻：月活躍使用者超過7億須另向Meta申請授權。
• 微調與散布由 云碩科技xCloudinfo 進行；不得用於非法用途；須遵守中華民國相關法律。

---

Built with Llama. 由 云碩科技xCloudinfo於自有AI算力資源池微調、散布；基底模型之權利屬Meta Platforms, Inc.。

關於云碩科技

云碩科技（xCloudinfo Corp. Limited）專注於地端私有化人工智慧解決方案。xVITA是云碩自研模型的品牌名稱。

其他模型請見 https://hf.co/xCloudinfo。

---

Llama-xReadable-11B-zhTW-OCR-v2 — English

Built with Llama

xCloudinfo (云碩科技)　·　Series: Meta Llama 3.2 Vision · xCloudinfo Traditional Chinese OCR

A Traditional Chinese (Taiwan) OCR model built on meta-llama/Llama-3.2-11B-Vision-Instruct and fine-tuned by xCloudinfo on its own AI compute pool. It focuses on text recognition for Taiwan scenarios — the mixed Chinese/English content of official documents, receipts, documents, and various kinds of notes — and can be deployed on-premises with no data leaving the site. xReadable: making documents readable.

A newer version is available

For new deployments please use Llama-xReadable-11B-zhTW-OCR-v3 — the font diversity has been expanded from 5 design families to 16, and printed-text recognition is better than this version across every metric. This page is kept for the reference of existing users.

What changed in v2 relative to v1: the real handwriting training samples were substantially expanded (including English-only samples for the first time), and long-form handwriting recognition improved substantially; short-form handwriting is on par with v1; printed/scanned recognition stays at a production-usable level. Positioning: printed/scanned document recognition is the production-usable mainstay; handwriting recognition is still being iterated.

Capability and measurements (real data, character-level CER, lower is better)

All evaluations compute the character error rate (CER) on real photographed/scanned or rendered document images.

Measured performance on unseen fonts

The 0.73% above is measured on an internal test set whose rendered fonts come from the same source as the training data. To measure true generalization, we validated separately with fonts never used in training:

This model generalizes well to unseen fonts: 0.00% on an unseen Song font, 2.78% on an unseen sans-serif font.

Erratum (2026-07-23): a previous version of this card reported "unseen Song font 25~28%" and used it to describe the model as generalizing poorly to fonts. That number was caused by an error in making the test image, not by model performance: face 0 of Songti.ttc is a Simplified face (Songti SC Black), which lacks Traditional characters such as "碩體學識測試義區號電話機驗證準確"; PIL ImageFont.truetype(path, size) takes index=0 by default, so the missing characters were silently rendered as blank (not as tofu boxes). The model correctly recognized the text actually present in the image; it was the ground truth used for comparison that was wrong. After remaking the test images using Songti TC Regular (index=7) from the same file, the measured result was 0.00%. Corrected here for the record.

A reminder for those running their own evaluations: after rendering a test image, confirm character by character that it is not blank before using it for measurement, otherwise you will mistake missing font characters for a model defect.

A stronger version: font diversity has been expanded to 16 design families in v3 (with 8 additional families kept entirely out of training for generalization measurement), and the unseen sans-serif font improved from 2.78% to 1.39%. For new deployments we recommend going straight to v3.

Honest note: the 64~66% on short handwriting is an "image difficulty floor", not a model ceiling

We cross-validated on the same set of real short handwriting samples using an open-source OCR of a completely different architecture (NVIDIA nemotron-ocr-v2, a detection + recognition pipeline, not a language model), and the per-image CER was almost identical to this model (for example, a faint pencil scanned page was about 88% for both, and a clear page was about 45% for both). Two systems with fundamentally different principles producing the same per-image results means the number is mainly determined by the legibility of the image itself (faint pencil, scrawl, low contrast = a physical limit), not by the weakness of a single model. For clearly written short handwriting, the measured result can reach around 40%; image quality is the key variable.

Uses and limitations

• Suitable for: on-premises OCR of Taiwan Traditional Chinese printed/scanned documents, document digitization, preprocessing for receipt and note recognition; transcription of long-form handwritten notes (clearly improved in v2).
• Output normalization: this model outputs uniformly in Traditional Chinese (the training labels use Traditional normalization); when the input contains Simplified characters it will still output the corresponding Traditional.
• Limitations:
  • Font diversity: this version's synthetic training corpus covers 7 fonts; the measured result on unseen fonts is 0.00%–2.78%. v3 has been expanded to 16 design families and performs better.
  • Handwriting recognition is still being iterated; scrawled, faint, or low-contrast handwriting may produce errors, so please have a human review important uses.
  • English-only passages are included in training but the sample volume is still limited.
  • For tables/complex layouts, we recommend splitting into lines/cells before recognition.
  • The output is for recognition reference only and should not be the sole basis for critical decisions in law, medicine, finance, etc.

Format and compatibility

This repo provides bf16 safetensors (the native transformers/vLLM format) and does not provide GGUF: llama.cpp does not yet officially support conversion/loading of Llama 3.2 Vision's mllama architecture, so llama.cpp / LM Studio / Ollama cannot currently run this model; transformers and vLLM can load it natively. An officially compatible GGUF will be added once upstream support lands.

Hardware requirements

The bf16 weights are about 21GB; for inference we recommend a single GPU with ≥ 24GB VRAM (RTX 3090/4090, A10, L4, A100, etc.), or splitting across multiple cards with device_map="auto"; if memory is insufficient you can quantize to int8/int4 with bitsandbytes.

Usage (transformers)

from transformers import MllamaForConditionalGeneration, AutoProcessor
from PIL import Image
import torch

model_id = "xCloudinfo/Llama-xReadable-11B-zhTW-OCR-v2"
model = MllamaForConditionalGeneration.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
).eval()
processor = AutoProcessor.from_pretrained(model_id)

image = Image.open("your_document.jpg").convert("RGB")
prompt = "請辨識圖片中的所有文字，逐字完整輸出，只輸出文字本身。"
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Usage (vLLM)

vllm serve xCloudinfo/Llama-xReadable-11B-zhTW-OCR-v2 --dtype bfloat16 --max-model-len 4096

License and source statement (Llama 3.2 Community License)

This model is a derivative of Meta Llama 3.2, and distribution is subject to the Llama 3.2 Community License Agreement:

• Built with Llama. This model is built with Meta Llama 3.2.
• Base model: meta-llama/Llama-3.2-11B-Vision-Instruct (Meta Platforms, Inc.).
• Model naming: as required by the license, the derivative model's name begins with "Llama".
• License text: this distribution includes a copy of the Llama 3.2 Community License Agreement (see LICENSE in this repo); the license copy must be provided and this statement retained.
• Acceptable Use Policy: use must comply with Meta's Acceptable Use Policy.
• EU restriction: the Llama 3.2 license provides that individuals or companies domiciled or with a principal place of business in the EU are not granted the license to use the multimodal capabilities included in Llama 3.2 multimodal models.
• Large-scale service threshold: more than 700 million monthly active users requires a separate license from Meta.
• Fine-tuning and distribution are by xCloudinfo (云碩科技); it must not be used for illegal purposes; and it must comply with the relevant laws of the Republic of China (Taiwan).

---

Built with Llama. Fine-tuned and distributed by xCloudinfo (云碩科技) on its own AI compute pool; rights in the base model belong to Meta Platforms, Inc.

About xCloudinfo

xCloudinfo Corp. Limited (云碩科技) builds on-premises, private AI solutions. xVITA is xCloudinfo's in-house model brand.

More models: https://hf.co/xCloudinfo