yecsASR-omni-ctc-small

yecsASR-omni-ctc-small

⚠️ Experimental research model. Released for experimentation and as a downstream demonstration of the YECS corpus — not production-ready. See Limitations.

A compact (~330M-param) automatic speech recognition model for Yoruba–English intra-sentential code-switching, finetuned from Meta's Omnilingual ASR CTC model on the Yoruba-English Code-Switching (YECS) Corpus by LyngualLabs. It transcribes naturally code-switched Yoruba/English speech while preserving Yoruba tone diacritics (e.g. ọ̀, ṣ, á/à).

Highlights

• Base: omniASR_CTC_300M — wav2vec2 encoder + CTC head (no LLM decoder), ~330M params, 1.3GB.
• Data: ~95h of YECS code-switched speech (16kHz mono).
• Validation: WER 26.85 / UER 8.62 on a held-out YECS dev slice.
• Light & fast: ~8GB GPU during training; CTC inference runs faster than real-time.

Intended use

Transcribing conversational Yoruba-English code-switched audio (16kHz mono). For research, prototyping, and as a baseline/demo for the YECS Utility initiative.

Results

Held-out YECS dev slice (429 utterances). Best checkpoint = step 20,000.

Validation WER over training: 47.7 (1k) → 37.4 (5k) → 28.8 (13k) → 27.3 (18k) → 26.85 (20k).

Training

• Base model: omniASR_CTC_300M (Meta Omnilingual ASR, Apache-2.0)
• Dataset: YECS train split, ~95h, 16kHz mono; transcripts NFC-normalized + lowercased
• Steps: 20,000 (~2 hours on a single GPU, ~8GB used)
• Precision / objective: bf16 mixed precision, CTC criterion
• Optimizer: lr 1e-5, gradient accumulation 4, length-based batching
• Tokenizer: omniASR_tokenizer_v1 (character tokenizer)
• Framework: fairseq2 via the omnilingual_asr training recipe

⚠️ Limitations

• Experimental, single-corpus finetune — expect errors, especially on the English side of code-switches (e.g. electricity → electricy) and on proper names.
• Output is lowercase and unpunctuated — this is normal for ASR (the tokenizer models only spoken characters + tone marks). This checkpoint was trained before a punctuation-stripping fix, so raw output may contain a ⁇ (U+2047) unknown token where punctuation occurred — strip it as shown below.
• Trained and evaluated only on YECS-domain audio; may not generalize to other accents, recording conditions, or domains.
• Not a transformers model — load it with the omnilingual_asr library (below), not AutoModel.from_pretrained.

Usage

git clone https://github.com/facebookresearch/omnilingual-asr && cd omnilingual-asr
pip install -e . && pip install huggingface_hub

import re, torch
from huggingface_hub import hf_hub_download
from fairseq2.models.hub import load_model
from fairseq2.data.tokenizers.hub import load_tokenizer
from omnilingual_asr.models.inference.pipeline import ASRInferencePipeline

model = load_model("omniASR_CTC_300M", dtype=torch.bfloat16)
sd = torch.load(hf_hub_download("LyngualLabs/yecsASR-omni-ctc-small", "model.pt"), map_location="cpu")
model.load_state_dict(sd, strict=False)        # if this errors, try sd["model"]
tok = load_tokenizer("omniASR_tokenizer_v1")
pipe = ASRInferencePipeline(None, model=model, tokenizer=tok)

def clean_pred(t):                             # drop the unknown token (U+2047) left where
    return re.sub(r"\s+", " ", t.replace("\u2047", "")).strip()  # punctuation used to be

preds = pipe.transcribe(["sample.wav"])        # 16kHz mono; CTC ignores the lang arg
print([clean_pred(p) for p in preds])

Files

• model.pt — finetuned weights (fairseq2 state dict)
• omniASR_tokenizer_v1.model — character tokenizer (also auto-downloadable by name)
• model.yaml, config.yaml — architecture + training config

License & data

• Model weights derive from omniASR_CTC_300M (Apache-2.0, Meta Omnilingual ASR).
• Training data: the YECS Corpus (LyngualLabs), licensed NOODL-1.0 (Nwulite Obodo Open Data Licence). Review its terms before redistribution or commercial use.

Acknowledgements

• LyngualLabs — YECS Yoruba-English Code-Switching Corpus.
• Meta AI — Omnilingual ASR (base model + training recipe).