Downloads · 30 days
500
8% of all-time downloads
xCloudinfo/Gemma-4-E4B-xReadable-zhTW-OCR-v1
Gemma-4-E4B-xReadable-zhTW-OCR-v1 is a image-text-to-text model from xCloudinfo. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
云碩科技 · xCloudinfo · 系列:Google Gemma 4 · 云碩繁體中文OCR
Downloads · 30 days
500
8% of all-time downloads
All-time downloads
6.1K
Public
Parameters
7.9B
22.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors15.9 GB · 72%
From the Hugging Face model README
云碩科技 · xCloudinfo · 系列:Google Gemma 4 · 云碩繁體中文OCR
以 google/gemma-4-E4B-it 為基底,由云碩科技在自有AI算力資源池上微調的繁體中文(台灣)OCR模型。專注台灣場景的文字辨識——公文、票據、文件與各類筆記的中英混寫內容,可地端部署、資料不外流。xReadable:讓文件被讀懂。(x = xCloudinfo品牌前綴,Read + able = 讓文件可被讀取。)
與Llama版(
Llama-xReadable)並存、各補一個部署場景:本模型基於Gemma 4 Vision,可轉GGUF,能直接跑在llama.cpp/Ollama/LM Studio(Llama 3.2 Vision版受mllama架構限制無GGUF)。授權亦更寬鬆(Gemma Terms,無歐盟多模態限制)。微調時凍結視覺編碼器,僅以LoRA訓練語言側,看圖能力承襲Gemma 4原生視覺塔。
本repo 同時內含兩種格式,依部署環境自行取用,無須另找GGUF repo:
| 用途 | 需下載的檔案 | 適用 |
|---|---|---|
| transformers / vLLM | model.safetensors + 設定/tokenizer檔 | 伺服器推論、微調、最高精度(bf16) |
| llama.cpp / Ollama / LM Studio(含看圖) | Gemma-4-E4B-xReadable-zhTW-OCR-v1-Q4_K_M.gguf(約5GB)+ Gemma-4-E4B-xReadable-zhTW-OCR-v1-mmproj-f16.gguf(約1GB) | 地端輕量、CPU可跑 |
GGUF看圖辨識必須同時載入
mmproj-f16(視覺投影器),只載主權重僅能處理純文字。量化GGUF精度略低於bf16 safetensors,關鍵用途建議用safetensors。
逐字元錯誤率(CER,越低越好)。評測字型與訓練字型完全不重疊——保留字型從建池起即切開,從未進入訓練。
| 量測 | 本模型(Gemma-4-E4B) | Llama-xReadable v3(參考) | nemotron-ocr-v2(NVIDIA,外部對照) |
|---|---|---|---|
| 未見字型·宋體(macOS Songti TC) | 0.00% | 0.00% | 2.78% |
| 未見字型·黑體(macOS STHeiti) | 0.00% | 1.39% | 9.72% |
| 保留字型集(8家族,200張) | 5.08% | 7.28% | — |
本模型在上述三項印刷OCR量測上均優於前一代Llama版與外部對照,且體積更小(8.0B vs 10.8B)。外部對照為 nvidia/nemotron-ocr-v2(偵測+辨識管線),相同影像、相同CER計算。
| 場景 | CER | 說明 |
|---|---|---|
| 清晰短篇 | 約40% | 實驗性,需人工複核 |
| 潦草/淡筆 | 64~88% | 影像難度地板,非模型天花板(見下) |
手寫的64~88% 主要由影像可辨識度決定:以架構完全不同的開源OCR交叉驗證,逐張CER與本系列模型幾乎一致(某淡鉛筆頁兩者皆約88%、某清晰頁皆約45%)。兩種原理迥異的系統得到相同的逐張結果,代表該數字由影像本身決定,而非單一模型的弱點。
訓練字型(16個設計家族):Noto Sans/Serif CJK、文鼎UMing/UKai、Chiron Hei/Sung/GoRound、Zev Hei、GenKiGothic/GenKiMin/GenWanMin、cwTeX明/楷/圓、jf-openhuninn、辰宇落雁體。 保留字型(8家族,永不進訓練):GenRyuMin、GenSekiGothic、GenSenRounded、cwTeX仿宋、EvilSung、LXGW文楷、芫荽Iansui、Cubic 11。 同一套設計的所有字重與正/斜體歸同一家族,避免同一設計同時出現在訓練與評測而高估泛化。
本repo同時提供 bf16 safetensors(transformers/vLLM)與 GGUF(llama.cpp/Ollama/LM Studio)。
from transformers import Gemma4ForConditionalGeneration, AutoProcessor
from PIL import Image
import torch
model_id = "xCloudinfo/Gemma-4-E4B-xReadable-zhTW-OCR-v1"
model = Gemma4ForConditionalGeneration.from_pretrained(
model_id, torch_dtype=torch.bfloat16, device_map="auto").eval()
processor = AutoProcessor.from_pretrained(model_id)
image = Image.open("your_document.jpg").convert("RGB")
prompt = "請辨識圖片中的所有文字,逐字完整輸出,只輸出文字本身。"
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# 文字+視覺:主權重 + 視覺投影器 mmproj
llama-server \
-m Gemma-4-E4B-xReadable-zhTW-OCR-v1-Q4_K_M.gguf \
--mmproj Gemma-4-E4B-xReadable-zhTW-OCR-v1-mmproj-f16.gguf \
--port 8080
Q4_K_M(約5GB)為量化主權重;mmproj-f16(約1GB)為視覺投影器,看圖辨識必須一併載入。本repo之GGUF已實測可在llama.cpp讀取圖片並輸出繁體中文;量化版辨識精度略低於bf16,關鍵用途建議用safetensors。
bf16約16GB;Q4_K_M GGUF約5GB,可於一般消費級GPU或純CPU執行。
本模型為Google Gemma的衍生模型,散布與使用須遵守 Gemma Terms of Use 及 Gemma Prohibited Use Policy:
google/gemma-4-E4B-it(Google)。訓練語料由云碩自有資料產線產生:自動化抓取權威/公開來源 → 去識別化 → 雙層品質閘門過濾 → 合成影像渲染(開放授權字型,SIL OFL 1.1/jf開放授權等,僅用於產生影像,本repo不散布字型檔)。全流程於自有AI算力資源池執行。
云碩科技(xCloudinfo Corp. Limited)專注於地端私有化人工智慧解決方案。xVITA是云碩自研模型的品牌名稱。
其他模型請見 https://hf.co/xCloudinfo。
由 云碩科技xCloudinfo於自有AI算力資源池微調、散布;基底模型之權利屬Google。Gemma is provided under and subject to the Gemma Terms of Use.
xCloudinfo (云碩科技) · Series: Google Gemma 4 · xCloudinfo Traditional Chinese OCR
A Traditional Chinese (Taiwan) OCR model built on google/gemma-4-E4B-it and fine-tuned by xCloudinfo on its own AI compute pool. It focuses on text recognition for Taiwan scenarios — the mixed Chinese/English content of official documents, receipts, documents, and various kinds of notes — and can be deployed on-premises with no data leaving the site. xReadable: making documents readable. (x = the xCloudinfo brand prefix; Read + able = making documents readable.)
Coexists with the Llama version (
Llama-xReadable), each covering a different deployment scenario: this model is based on Gemma 4 Vision and can be converted to GGUF, so it runs directly onllama.cpp/Ollama/LM Studio(the Llama 3.2 Vision version has no GGUF because of themllamaarchitecture restriction). Its license is also more permissive (Gemma Terms, with no EU multimodal restriction). The vision encoder was frozen during fine-tuning; only the language side was trained with LoRA, so image understanding is inherited from Gemma 4's native vision tower.
This repo contains both formats; take whichever fits your deployment environment, with no need to look for a separate GGUF repo:
| Use | Files to download | Suited to |
|---|---|---|
| transformers / vLLM | model.safetensors + config/tokenizer files | Server inference, fine-tuning, highest precision (bf16) |
| llama.cpp / Ollama / LM Studio (with image input) | Gemma-4-E4B-xReadable-zhTW-OCR-v1-Q4_K_M.gguf (about 5GB) + Gemma-4-E4B-xReadable-zhTW-OCR-v1-mmproj-f16.gguf (about 1GB) | Lightweight on-premises use; runs on CPU |
For image recognition with GGUF,
mmproj-f16(the vision projector) must be loaded as well; loading only the main weights handles plain text only. The quantized GGUF is slightly less accurate than the bf16 safetensors; for critical uses we recommend the safetensors.
Character error rate (CER, lower is better). The evaluation fonts do not overlap with the training fonts at all — the held-out fonts were split off when the font pool was built and have never entered training.
| Measurement | This model (Gemma-4-E4B) | Llama-xReadable v3 (reference) | nemotron-ocr-v2 (NVIDIA, external comparison) |
|---|---|---|---|
| Unseen font · Song (macOS Songti TC) | 0.00% | 0.00% | 2.78% |
| Unseen font · Hei (macOS STHeiti) | 0.00% | 1.39% | 9.72% |
| Held-out font set (8 families, 200 images) | 5.08% | 7.28% | — |
On all three of the printed-OCR measurements above, this model is better than the previous-generation Llama version and the external comparison, and it is smaller (8.0B vs 10.8B). The external comparison is nvidia/nemotron-ocr-v2 (a detection + recognition pipeline), on the same images with the same CER computation.
| Scenario | CER | Notes |
|---|---|---|
| Clear short text | about 40% | Experimental; requires human review |
| Scrawled / faint strokes | 64~88% | An image-difficulty floor, not a model ceiling (see below) |
The 64~88% on handwriting is mainly determined by the legibility of the image: cross-validated with an open-source OCR of a completely different architecture, the per-image CER was almost identical to this model series (a faint pencil page was about 88% for both, and a clear page about 45% for both). Two systems with fundamentally different principles producing the same per-image results means the number is determined by the image itself, not by the weakness of a single model.
Training fonts (16 design families): Noto Sans/Serif CJK, Arphic (文鼎) UMing/UKai, Chiron Hei/Sung/GoRound, Zev Hei, GenKiGothic/GenKiMin/GenWanMin, cwTeX Ming/Kai/Yuan (明/楷/圓), jf-openhuninn, Chenyu Luoyan (辰宇落雁體). Held-out fonts (8 families, never used in training): GenRyuMin, GenSekiGothic, GenSenRounded, cwTeX Fangsong (仿宋), EvilSung, LXGW WenKai (LXGW 文楷), Iansui (芫荽), Cubic 11. All weights and upright/italic variants of the same design are assigned to the same family, so that the same design never appears on both the training and evaluation sides and inflates the measured generalization.
This repo provides both bf16 safetensors (transformers/vLLM) and GGUF (llama.cpp/Ollama/LM Studio).
from transformers import Gemma4ForConditionalGeneration, AutoProcessor
from PIL import Image
import torch
model_id = "xCloudinfo/Gemma-4-E4B-xReadable-zhTW-OCR-v1"
model = Gemma4ForConditionalGeneration.from_pretrained(
model_id, torch_dtype=torch.bfloat16, device_map="auto").eval()
processor = AutoProcessor.from_pretrained(model_id)
image = Image.open("your_document.jpg").convert("RGB")
prompt = "請辨識圖片中的所有文字,逐字完整輸出,只輸出文字本身。"
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=768, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# 文字+視覺:主權重 + 視覺投影器 mmproj
llama-server \
-m Gemma-4-E4B-xReadable-zhTW-OCR-v1-Q4_K_M.gguf \
--mmproj Gemma-4-E4B-xReadable-zhTW-OCR-v1-mmproj-f16.gguf \
--port 8080
Q4_K_M (about 5GB) is the quantized main weights; mmproj-f16 (about 1GB) is the vision projector, and it must be loaded as well for image recognition. The GGUF in this repo has been tested to read images in llama.cpp and output Traditional Chinese; the quantized version's recognition accuracy is slightly lower than bf16, so for critical uses we recommend the safetensors.
bf16 is about 16GB; the Q4_K_M GGUF is about 5GB and can run on an ordinary consumer GPU or on CPU only.
This model is a derivative of Google Gemma, and its distribution and use are subject to the Gemma Terms of Use and the Gemma Prohibited Use Policy:
google/gemma-4-E4B-it (Google).The training corpus is produced by xCloudinfo's own data pipeline: automated collection from authoritative/public sources → de-identification → filtering through a two-layer quality gate → synthetic image rendering (openly licensed fonts, SIL OFL 1.1 / the jf open license, etc., used only to generate images; this repo does not distribute the font files). The entire process runs on xCloudinfo's own AI compute pool.
xCloudinfo Corp. Limited (云碩科技) builds on-premises, private AI solutions. xVITA is xCloudinfo's in-house model brand.
More models: https://hf.co/xCloudinfo
Fine-tuned and distributed by xCloudinfo (云碩科技) on its own AI compute pool; rights in the base model belong to Google. Gemma is provided under and subject to the Gemma Terms of Use.