whisper-medium-mla-cv11

Whisper-Medium-MLA (11 languages) — 62.5% smaller decode KV-cache

openai/whisper-medium (769M) with decoder self-attention converted MHA→MLA (Whisper-MLA, arXiv:2603.00563), recovery-fine-tuned on 11 languages of the CC0 Whispered corpus (32k clips/lang). The largest absolute cache savings of the released family — medium's per-stream decode cache is ~2.7× small's, so the 62.5% cut buys the most bytes here. For the family's top accuracy tier, see whisper-large-v3-mla.

from transformers import AutoModelForSpeechSeq2Seq
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-medium-mla-cv11", trust_remote_code=True)  # transformers==4.46.x

Results (CommonVoice-17 test, n=1500/lang; cost = paired vs identically-trained unconverted control)

Median cost ≈ +0.4; not significant on 6/11 languages (zh cost is significant on CER). † ka: the compression cost collapses to ns at medium, but ka itself stays unusable (80.6 WER; capacity was and is its problem).

The honest scaling note (preregistered prediction, half-held)

We predicted medium's cost at ≈+0.2 before training. Measured: ≈+0.4 — the size-scaling curve SATURATES (tiny ≈+1.9 → base ≈+1.0 → small ≈+0.4 → medium ≈+0.4; approximate per-size medians) rather than decaying to zero. The practical claim holds (all languages ≤+1.0, mostly ns); the mechanism reading is an irreducible truncation cost at this rank, not ever-growing redundancy absorption.

Matched control now published — verify the conversion cost yourself: burakaydinofficial/whisper-medium-cv11 (trained identically, minus the MHA→MLA conversion). Evaluate both with scripts/validate.py.

Limitations

• Costs are recovery-mix-specific: do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (French in the small model: +0.32 cv11 vs +1.22 24-lang). Not irreproducibility.
• What the 62.5% is (cache scope): it is the decode self-attention KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream total decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
• Requires trust_remote_code=True and transformers==4.46.x (custom MLA attention modules); not loadable in whisper.cpp / faster-whisper / CTranslate2.
• Language coverage is the 11 recovery languages — unseen scripts degrade (measured).
• Consumer-mic read-speech domain (real environmental noise included; SNR-ladder on the small tier showed cost flat to 0 dB); greedy-decode evals — beam-5 adds ~1-2 WER on both arms without changing the conversion cost (measured on the small tier).
• Training: 15k steps, warmup+cosine, encoder frozen both arms, dev-checkpoint-selected, bf16 (weights released as fp16).

FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)

ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified to be small on the small tier). Per-language absolute: en 9.6 · de 9.7 · es 7.3 · fr 11.0 · ru 12.6 · tr 14.4 · cy 41.4 · ar 20.8 · th 20.0 · zh-CN 11.1 · ka 92.4. (Full JSON: the release repo's eval artifacts.)