yecs-asr-whisper-plain

YECS Whisper-small — Plain Baseline

openai/whisper-small (244M) fine-tuned on the YECS Yoruba–English code-switching corpus for plain transcription (no language tags). It is the control for the language-tag-injection A/B study — its tagged twin, LyngualLabs/yecs-asr-whisper-lid, adds 99.54% per-word language ID at no WER cost.

Results (YECS full held-out test, 9,949 utterances)

A 5-epoch fine-tune on YECS brings whisper-small well below the previously reported Yoruba baseline.

Normalization: NFC → lowercase → strip Unicode P/S punctuation. Tone-aware keeps Yoruba tone diacritics (acute/grave/macron); tone-insensitive strips them but always keeps the phonemic under-dot (ọ, ẹ, ṣ). See the tagged model's card for the fuller note.

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor
import torch, librosa

repo = "LyngualLabs/yecs-asr-whisper-plain"
proc = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(repo, torch_dtype=torch.bfloat16).to("cuda").eval()

audio, _ = librosa.load("utterance.wav", sr=16000)
feats = proc.feature_extractor(audio, sampling_rate=16000, return_tensors="pt").input_features
ids = model.generate(feats.to("cuda", torch.bfloat16),
                     language="yoruba", task="transcribe", max_new_tokens=225)
print(proc.tokenizer.batch_decode(ids, skip_special_tokens=True)[0])

Training

• Base: openai/whisper-small · 5 epochs · lr 1e-5 · bf16 · effective batch 32 · 1×H200.
• Data: YECS train (80,013 utts / ~95.6 h), 16 kHz mono; best checkpoint by dev WER.
• Language conditioning yoruba, task transcribe.
• W&B: afroscale_ai_cmu_africa/yecs-lid, run whisper-plain.

Intended use & limitations

Research baseline for Yoruba–English code-switching ASR. Trained on YECS read/prompted speech; performance may drop on far-field, noisy, or spontaneous conversational audio. Two languages only (yo, en).

Related

• Tag-injection variant (joint ASR + 99.54% per-word LID): LyngualLabs/yecs-asr-whisper-lid
• Omnilingual variants: yecs-asr-ctc-lid · yecs-asr-llm-lid
• Code & writeup: https://github.com/osinkolu/yecs-asr-benchmark (tag-injection/)

Evaluation note

Scored on the full held-out YECS test split (9,949 utts); test audio is the canonical Mozilla/MDC release. Test reference labels are held privately by LyngualLabs, so metrics are internal and not independently reproducible.

License

Apache-2.0 (inherited from openai/whisper-small). YECS corpus terms apply to the training data.