Downloads · 30 days
55
2% of all-time downloads
xCloudinfo/Whisper-Large-v3-Turbo-xVITA-zhTW
Whisper-Large-v3-Turbo-xVITA-zhTW is a automatic speech recognition model from xCloudinfo. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as mit.
台灣華語(繁體中文)語音辨識,特別針對真實會議語音優化的輕量快速版本。 以 openai/whisper-large-v3-turbo 為底,用台灣在地語料微調。
Downloads · 30 days
55
2% of all-time downloads
All-time downloads
2.8K
Public
Parameters
809M
3.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.bin1.6 GB · 50%
From the Hugging Face model README
台灣華語(繁體中文)語音辨識,特別針對真實會議語音優化的輕量快速版本。
以 openai/whisper-large-v3-turbo 為底,用台灣在地語料微調。
xVITA = Verified Intelligence for Taiwan(可驗證的台灣智慧),云碩科技(xCloudinfo)自建語音/語言模型家族。
會議語音轉寫最強、速度快4倍;單句朗讀的文字準度略遜家族中的large-v3版本。
真實台灣商務會議錄音,自發口語、含填充詞與中英夾雜。此測試集未參與訓練,也未用於選擇檢查點。
| 模型 | 內容CER | 耗時 |
|---|---|---|
whisper-large-v3-turbo(底模) | 19.40% | 49s |
| 本模型 | 12.85% | 55s |
| Whisper-Large-v3-xVITA-zhTW | 13.25% | 221s |
在最貼近實務的會議場景,本模型準度最佳,且比large-v3版本快4倍。
| 模型 | 內容CER | 實際CER<sup>註</sup> | 耗時 |
|---|---|---|---|
whisper-large-v3-turbo(底模) | 7.13% | 35.04% | 198s |
| 本模型 | 5.94% | 5.85% | 195s |
whisper-large-v3(底模) | 5.94% | 33.26% | 923s |
| Whisper-Large-v3-xVITA-zhTW | 5.58% | 6.63% | 856s |
<sup>註</sup> 實際CER=模型原始輸出直接對繁體標註比對。底模因輸出簡體而大幅扣分,這欄反映「能不能直接用、不必後處理」。
朗讀語料上本模型與 whisper-large-v3 底模同分,略遜large-v3微調版0.36個百分點。
| 模型 | 轉出字數 |
|---|---|
| 本模型 | 248字 |
| Whisper-Large-v3-xVITA-zhTW | 247字 |
長音檔內容涵蓋量與large-v3版本相當,無截斷。
底模在64秒測試音檔上出現 37種簡體字;本模型降至 0–1種。
generation_config.json 另封禁110個純簡體token,解碼空間出不了簡體,可直接使用不需OpenCC後處理。
openai/whisper-large-v3-turbo(MIT)會議語料的音檔與逐字稿以自建管線重新切段,並逐段以ASR交叉核對音文是否對得上, 僅保留通過者(通過率62.9%)。
這一步不是可選的:我們曾有一個內部版本使用未經對齊驗證的同批會議語料, 結果在會議測試集上CER 30.69% —— 比未微調的底模(19.40%)還差11個百分點。 錯位的訓練資料不是「沒有幫助」,而是主動傷害模型。此處據實記載,供同類工作參考。
語料另經去識別化處理;訓練於云碩自有AI算力資源池完成,資料未離開自有場域。
from transformers import pipeline
asr = pipeline("automatic-speech-recognition",
model="xCloudinfo/Whisper-Large-v3-Turbo-xVITA-zhTW",
device=0, chunk_length_s=30)
print(asr("meeting.wav", generate_kwargs={"language": "zh", "task": "transcribe"})["text"])
倉庫同時提供 CTranslate2格式(ct2/),可直接餵給 faster-whisper:
from faster_whisper import WhisperModel
model = WhisperModel("ct2", device="cuda", compute_type="float16")
segments, _ = model.transcribe("meeting.wav", language="zh", without_timestamps=True)
MIT,與底模 openai/whisper-large-v3-turbo 一致。
本模型與TAIDE專案無關,未使用TAIDE作為教師模型、未包含TAIDE授權語料。
@misc{xvita-whisper-turbo-zhtw-2026,
title = {Whisper-Large-v3-Turbo-xVITA-zhTW},
author = {xCloudinfo},
year = {2026},
url = {https://huggingface.co/xCloudinfo/Whisper-Large-v3-Turbo-xVITA-zhTW}
}
云碩科技(xCloudinfo Corp. Limited)專注於地端私有化人工智慧解決方案。xVITA是云碩自研模型的品牌名稱。
其他模型請見 https://hf.co/xCloudinfo。
Taiwanese Mandarin (Traditional Chinese) speech recognition, a lightweight, fast version optimized especially for real meeting speech. Built on openai/whisper-large-v3-turbo and fine-tuned with local Taiwan corpora.
xVITA = Verified Intelligence for Taiwan, xCloudinfo's (云碩科技) in-house speech/language model family.
Best for meeting-speech transcription, and 4x faster; text accuracy on single-sentence read speech is slightly behind the large-v3 version in the family.
Real Taiwan business meeting recordings, spontaneous speech with fillers and mixed Chinese/English. This test set did not take part in training and was not used to select checkpoints.
| Model | Content CER | Time |
|---|---|---|
whisper-large-v3-turbo (base) | 19.40% | 49s |
| This model | 12.85% | 55s |
| Whisper-Large-v3-xVITA-zhTW | 13.25% | 221s |
In the meeting scenario, which is closest to practice, this model has the best accuracy and is 4x faster than the large-v3 version.
| Model | Content CER | Actual CER<sup>†</sup> | Time |
|---|---|---|---|
whisper-large-v3-turbo (base) | 7.13% | 35.04% | 198s |
| This model | 5.94% | 5.85% | 195s |
whisper-large-v3 (base) | 5.94% | 33.26% | 923s |
| Whisper-Large-v3-xVITA-zhTW | 5.58% | 6.63% | 856s |
<sup>†</sup> Actual CER = the model's raw output compared directly against the Traditional Chinese labels. The base models are heavily penalized because they output Simplified; this column reflects "can it be used directly, without post-processing".
On the read-speech corpus this model ties the whisper-large-v3 base model and is 0.36 percentage points behind the large-v3 fine-tuned version.
| Model | Characters transcribed |
|---|---|
| This model | 248 characters |
| Whisper-Large-v3-xVITA-zhTW | 247 characters |
The content coverage on long audio is comparable to the large-v3 version, with no truncation.
On the 64-second test audio the base model produced 37 kinds of Simplified characters; this model drops to 0–1 kinds.
generation_config.json additionally bans 110 Simplified-only tokens, so the decoding space cannot produce Simplified, and it can be used directly without OpenCC post-processing.
openai/whisper-large-v3-turbo (MIT)The audio and transcripts of the meeting corpus were re-segmented with an in-house pipeline, and each segment was cross-checked by ASR to see whether the audio and text matched, keeping only those that passed (62.9% pass rate).
This step is not optional: we once had an internal version that used the same batch of meeting corpus without alignment validation, and it scored 30.69% CER on the meeting test set — 11 percentage points worse than the un-fine-tuned base model (19.40%). Misaligned training data does not merely "fail to help"; it actively harms the model. Recorded here faithfully for the reference of similar work.
The corpus was also de-identified; training was completed on xCloudinfo's own AI compute pool, and the data did not leave the owned site.
from transformers import pipeline
asr = pipeline("automatic-speech-recognition",
model="xCloudinfo/Whisper-Large-v3-Turbo-xVITA-zhTW",
device=0, chunk_length_s=30)
print(asr("meeting.wav", generate_kwargs={"language": "zh", "task": "transcribe"})["text"])
The repo also provides a CTranslate2 format (ct2/) that can be fed directly to faster-whisper:
from faster_whisper import WhisperModel
model = WhisperModel("ct2", device="cuda", compute_type="float16")
segments, _ = model.transcribe("meeting.wav", language="zh", without_timestamps=True)
MIT, consistent with the base model openai/whisper-large-v3-turbo.
This model is unrelated to the TAIDE project, did not use TAIDE as a teacher model, and does not include TAIDE-licensed corpus.
@misc{xvita-whisper-turbo-zhtw-2026,
title = {Whisper-Large-v3-Turbo-xVITA-zhTW},
author = {xCloudinfo},
year = {2026},
url = {https://huggingface.co/xCloudinfo/Whisper-Large-v3-Turbo-xVITA-zhTW}
}
xCloudinfo Corp. Limited (云碩科技) builds on-premises, private AI solutions. xVITA is xCloudinfo's in-house model brand.
More models: https://hf.co/xCloudinfo