whisper-small-cv11

Whisper-Small-CV11 — 11-language fine-tune, standard architecture, loads anywhere

A plain openai/whisper-small (unmodified architecture) fine-tuned on 11 languages of the CC0 Whispered corpus (32k clips/language, ~350 h). No custom code — works in transformers, faster-whisper, CTranslate2, whisper.cpp conversions, anywhere a Whisper checkpoint works.

It is also the scientific control for the WhisperMLA release (trained identically to the MLA variants, minus the conversion) — which makes its numbers directly comparable to theirs.

from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-small-cv11")   # no trust_remote_code needed
processor = AutoProcessor.from_pretrained("burakaydinofficial/whisper-small-cv11")

Results (CommonVoice-17 test, n=1500/lang, greedy, Whisper normalization + Arabic folding)

Better than stock on 10 of 11 languages (−0.5 on en). Encoder frozen during fine-tuning (the multilingual encoder is preserved); 15,000 steps, warmup+cosine, dev-checkpoint-selected; bf16.

Limitations

• Coverage = these 11 languages; fine-tuning erodes Whisper's other ~88 (unseen scripts most — measured in the accompanying study). Use stock Whisper for languages far from this set.
• Georgian (ka) reported for transparency; not usable at the whisper-small tier.
• Read-speech domain (CommonVoice + FLEURS-validated); not benchmarked on far-field/telephony.

Acoustic conditions of the evaluation

Evaluated on crowdsourced consumer-microphone recordings with real environmental noise — traffic, room reverb, variable devices — CommonVoice's native conditions, not studio audio. The numbers above already include that heterogeneity. Not yet benchmarked: far-field, telephony (8 kHz), overlapping speech.

(Note: this is the unconverted control — it has no MHA→MLA conversion, so "compression cost" does not apply to it. The SNR-robustness-of-compression result lives on the MLA cards, where it is measured as MLA-vs-this-control.)