w2v-bert-2.0-luganda-main-best-2

w2v-bert-2.0-luganda-main-best-2

A Luganda automatic speech recognition (ASR) model, fine-tuned from facebook/w2v-bert-2.0 on a combined Luganda speech corpus. This is a variant of keystats/w2v-bert-2.0-luganda-main-best with a different train/validation split of the WAXAL data — see Difference from -main-best below.

Model description

facebook/w2v-bert-2.0 — a large-scale, multilingual self-supervised speech encoder pretrained with a BERT-style masked prediction objective — is used as the backbone, with a from-scratch character-level CTC (Connectionist Temporal Classification) head fine-tuned specifically for Luganda.

Text casing note: like -main-best, this model's vocabulary was built with NFKC-only normalization — case, punctuation, and diacritics from the source transcriptions are preserved as-is. This matches how the WAXAL competition (Zindi) actually scores submissions: raw, unnormalized WER/CER, where case and punctuation mismatches count as errors.

Difference from -main-best

This checkpoint uses a different split of the WAXAL data than -main-best:

Everything else — base model, architecture, processor/tokenizer construction, hyperparameters, data filtering, and seed — is unchanged from -main-best. See below for the full details.

Training data

Training pool (train split only from each source, deduplicated by audio hash and by transcription-within-source, unless noted otherwise):

Note: keystats/luganda_asr_dataset (~230k rows, used in the -main sibling's training) was deliberately excluded, for the same reason as in -main-best — dropping it makes it easier to train more epochs.

None of the additional sources overlap with WAXAL's own train/validation/test split boundaries, so pooling every split from them carries no evaluation leakage risk.

Because the WAXAL validation split is now part of the training pool, it is not a valid held-out benchmark for this checkpoint. The WAXAL test split was used as the training-time validation set (for early stopping / best-checkpoint selection), so it is also not a clean independent benchmark for this checkpoint. Evaluate this model on a genuinely held-out set if you need an unbiased score.

Training procedure

• Base model: facebook/w2v-bert-2.0
• Architecture: Wav2Vec2BertForCTC, add_adapter=True
• Processor: Wav2Vec2BertProcessor — SeamlessM4TFeatureExtractor for audio features + a Wav2Vec2CTCTokenizer built from scratch on the combined training + validation transcriptions (character-level vocabulary, case and punctuation preserved, NFKC Unicode normalization only, | as the word delimiter, [PAD] doubling as the CTC blank token)
• Sample rate: 16 kHz mono
• Hardware: single RTX PRO 6000
• Epochs: 10 (with early stopping, patience 5, on validation WER)

• Data filtering: clips whose transcript is too long for CTC to align within the available encoder output length ("CTC-impossible" clips, roughly output_steps < 2 * label_length) are dropped from both train and validation before training
• Seed: 42 (deterministic — same seed for Python/NumPy/PyTorch/CUDA)

Evaluation results

Evaluation numbers for this checkpoint are not available yet in this README — they'll be added once a proper held-out evaluation has been run (see the note in Training data on why WAXAL's validation/test splits aren't clean benchmarks for this model).

How to use

Two ways to use this model, depending on your needs:

• Option 1 — model alone (greedy decoding): faster, no extra dependencies, slightly lower accuracy.
• Option 2 — model + KLM (recommended): requires pyctcdecode + KenLM, noticeably higher accuracy via beam-search decoding with a matching language model.

Option 1 — model alone (greedy decoding)

import torch
import librosa
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor

MODEL_ID = "keystats/w2v-bert-2.0-luganda-main-best-2"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]

print(transcription)  # cased, punctuated Luganda text (to whatever extent seen in training)

Option 2 — model + KLM (recommended, higher accuracy)

A companion n-gram KenLM language model, trained on the same raw (cased, punctuated) text convention as this ASR model, is available at keystats/waxal-kenlm-models-best (luganda/luganda_5gram_correct-best.arpa).

Important: use the matching KLM variant for whichever ASR checkpoint you're using — this model pairs with keystats/waxal-kenlm-models-best (raw/cased text), while the -main checkpoint pairs with the separate keystats/waxal-kenlm-models repo (normalized/lowercase text). Mixing a raw-text ASR model with a normalized-text KLM (or vice versa) will cause a vocabulary mismatch during decoding.

# pip install pyctcdecode
# pip install https://github.com/kpu/kenlm/archive/master.zip

import torch
import librosa
from huggingface_hub import hf_hub_download
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor
from pyctcdecode import build_ctcdecoder

MODEL_ID = "keystats/w2v-bert-2.0-luganda-main-best-2"
KLM_REPO_ID = "keystats/waxal-kenlm-models-best"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

klm_path = hf_hub_download(repo_id=KLM_REPO_ID, repo_type="dataset",
                            filename="luganda/luganda_5gram_correct-best.arpa")

def build_vocab_list(tokenizer, vocab_size):
    vocab_dict = tokenizer.get_vocab()
    vocab_list = [None] * vocab_size
    for tok, idx in sorted(vocab_dict.items(), key=lambda kv: kv[1]):
        if idx < vocab_size:
            vocab_list[idx] = tok
    pad_id = tokenizer.pad_token_id
    if pad_id is not None and pad_id < len(vocab_list):
        vocab_list[pad_id] = ""
    word_delim = getattr(tokenizer, "word_delimiter_token", None)
    if word_delim:
        delim_id = vocab_dict.get(word_delim)
        if delim_id is not None:
            vocab_list[delim_id] = " "
    return vocab_list

vocab_list = build_vocab_list(processor.tokenizer, model.config.vocab_size)
decoder = build_ctcdecoder(
    vocab_list,
    kenlm_model_path=klm_path,
    alpha=0.5,   # LM weight -- tune against your own validation set
    beta=0.7,    # word insertion bonus -- tune against your own validation set
)

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

transcription = decoder.decode(logits.cpu().numpy()[0], beam_width=100)
print(transcription)

Note on alpha/beta: the values above are starting points, not universal defaults — grid-search them against your own labeled validation set, since optimal weights depend on your specific audio domain.

Intended uses & limitations

• Intended for transcribing spoken Luganda audio into cased, punctuated text (to whatever extent case/punctuation exist in the training transcriptions).
• As a CTC-based model, it assumes single-speaker, forward-only audio and has no mechanism for overlapping speech from multiple speakers.
• Trained on a mix of WAXAL and two community-contributed Luganda datasets; acoustic conditions, recording quality, and dialectal coverage reflect that combined pool, not any single controlled source.
• Because WAXAL's validation split was folded into training and its test split was used for training-time model selection, neither split gives an unbiased read on this model's real-world accuracy — treat any score computed on them with that caveat in mind.
• Raw-text WER/CER (with case and punctuation counted as errors) will read higher than a normalized-text comparison of the same underlying transcription quality — this is expected and matches how the source competition (Zindi/WAXAL) actually scores submissions.

Related checkpoints

• keystats/w2v-bert-2.0-luganda-main-best — same setup, but with WAXAL's validation and test splits kept fully held out for evaluation instead of used in/for training
• keystats/w2v-bert-2.0-luganda-main — same base model and general training procedure, trained on lowercased text instead, and additionally includes keystats/luganda_asr_dataset in its training pool

Citation

If you use this model, please cite the training/fine-tuning work and the underlying datasets:

@misc{keystats_wav2vec2bert_luganda_best2,
  title={w2v-bert-2.0-luganda-main-best-2: A Luganda ASR model fine-tuned from facebook/w2v-bert-2.0 on raw text},
  author={keystats},
  year={2026},
  howpublished={\url{https://huggingface.co/keystats/w2v-bert-2.0-luganda-main-best-2}}
}

@misc{waxal,
  title={WAXAL: A Multilingual African Speech Dataset},
  author={Google},
  howpublished={\url{https://huggingface.co/datasets/google/WaxalNLP}}
}

@misc{farmerline_luganda,
  title={luganda\_dataset\_2.0},
  author={FarmerlineML},
  howpublished={\url{https://huggingface.co/datasets/FarmerlineML/luganda_dataset_2.0}}
}

@misc{bateesa_luganda_tts,
  title={luganda-tts-toby},
  author={Bateesa},
  howpublished={\url{https://huggingface.co/datasets/Bateesa/luganda-tts-toby}}
}

@inproceedings{w2vbert2,
  title={Seamless: Multilingual Expressive and Streaming Speech Translation},
  author={Seamless Communication and others},
  year={2023},
  howpublished={\url{https://huggingface.co/facebook/w2v-bert-2.0}}
}