w2v-bert-2.0-shona-main-best-2

w2v-bert-2.0-shona-main-best-2

A Shona automatic speech recognition (ASR) model, fine-tuned from facebook/w2v-bert-2.0 on a combined Shona speech corpus pooled from five sources (see Training data below).

Model description

facebook/w2v-bert-2.0 — a large-scale, multilingual self-supervised speech encoder pretrained with a BERT-style masked prediction objective — is used as the backbone, with a from-scratch character-level CTC (Connectionist Temporal Classification) head fine-tuned specifically for Shona.

Text casing note: training targets were kept in their raw, cased form — the vocabulary was built directly from the original transcriptions, punctuation and casing included, rather than lowercased first. This checkpoint follows the same cased-text convention as keystats/w2v-bert-2.0-shona-main-best, but differs from it in how the WAXAL splits were used during training — see Training data below.

Training data

Important split difference from the -best sibling checkpoint: in that earlier run, WAXAL's validation split was held out untouched as the evaluation benchmark. In this run, WAXAL's validation split was instead folded into the training pool, and WAXAL's test split was used as the validation set during training (for early stopping / best-checkpoint selection). As a result, there is no untouched WAXAL Shona split held back from this model — any WER/CER reported against WAXAL validation or test for this checkpoint reflects data the model was trained or model-selected on, not a clean benchmark.

Deduplication by audio hash and by transcription-within-source was applied across the combined training pool.

Training procedure

• Base model: facebook/w2v-bert-2.0
• Architecture: Wav2Vec2BertForCTC, add_adapter=True
• Processor: Wav2Vec2BertProcessor — SeamlessM4TFeatureExtractor for audio features + a Wav2Vec2CTCTokenizer built from scratch on the combined training + validation transcriptions (character-level vocabulary, raw/cased text, | as the word delimiter, [PAD] doubling as the CTC blank token)
• Sample rate: 16 kHz mono
• Epochs: 5 (with early stopping, patience 5, on validation WER)
• Effective batch size: 32 (per-device batch size 4 × gradient accumulation 8)
• Learning rate: 3e-5, cosine schedule, 10% warmup
• Precision: fp16, gradient checkpointing enabled
• Regularization: attention/hidden/feature-projection dropout 0.05
• Data filtering: clips whose transcript is too long for CTC to align within the available encoder output length ("CTC-impossible" clips, roughly output_steps < 2 * label_length) are dropped from both train and validation before training
• Seed: 42 (deterministic — same seed for Python/NumPy/PyTorch/CUDA)

Evaluation results

Because WAXAL test was used as the training-time validation set for this checkpoint (see Training data), there is no held-out WAXAL Shona split left to report clean benchmark numbers against. Adding a KenLM (keystats/waxal-kenlm-models-best) on top of greedy decoding gives a consistent, meaningful improvement in practice, as with the other checkpoints in this family, but any absolute WER/CER figures for this specific model should be treated as in-sample rather than benchmark numbers. If you need a clean apples-to-apples WAXAL benchmark score, use keystats/w2v-bert-2.0-shona-main-best instead, which kept WAXAL validation untouched.

How to use

Two ways to use this model, depending on your needs:

• Option 1 — model alone (greedy decoding): faster, no extra dependencies, slightly lower accuracy.
• Option 2 — model + KLM (recommended): requires pyctcdecode + KenLM, noticeably higher accuracy via beam-search decoding with a matching language model.

Option 1 — model alone (greedy decoding)

import torch
import librosa
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor

MODEL_ID = "keystats/w2v-bert-2.0-shona-main-best-2"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]

print(transcription)  # cased, punctuated Shona text

Option 2 — model + KLM (recommended, higher accuracy)

A companion n-gram KenLM language model, trained on the same raw/cased text convention as this ASR model, is available at keystats/waxal-kenlm-models-best (shona/shona_5gram_correct.arpa). Pairing this ASR model with its matching KLM via beam-search decoding gives a significant accuracy improvement over greedy decoding alone.

Important: use the matching KLM variant for whichever ASR checkpoint you're using — this model pairs with the cased keystats/waxal-kenlm-models-best repo, while the normalized (lowercased) keystats/w2v-bert-2.0-shona-main checkpoint pairs with the separate keystats/waxal-kenlm-models repo instead. Mixing a cased-text ASR model with a normalized-text KLM (or vice versa) will cause a vocabulary mismatch during decoding.

# pip install pyctcdecode
# pip install https://github.com/kpu/kenlm/archive/master.zip

import torch
import librosa
from huggingface_hub import hf_hub_download
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor
from pyctcdecode import build_ctcdecoder

MODEL_ID = "keystats/w2v-bert-2.0-shona-main-best-2"
KLM_REPO_ID = "keystats/waxal-kenlm-models-best"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

klm_path = hf_hub_download(repo_id=KLM_REPO_ID, repo_type="dataset",
                            filename="shona/shona_5gram_correct.arpa")

def build_vocab_list(tokenizer, vocab_size):
    vocab_dict = tokenizer.get_vocab()
    vocab_list = [None] * vocab_size
    for tok, idx in sorted(vocab_dict.items(), key=lambda kv: kv[1]):
        if idx < vocab_size:
            vocab_list[idx] = tok
    pad_id = tokenizer.pad_token_id
    if pad_id is not None and pad_id < len(vocab_list):
        vocab_list[pad_id] = ""
    word_delim = getattr(tokenizer, "word_delimiter_token", None)
    if word_delim:
        delim_id = vocab_dict.get(word_delim)
        if delim_id is not None:
            vocab_list[delim_id] = " "
    return vocab_list

vocab_list = build_vocab_list(processor.tokenizer, model.config.vocab_size)
decoder = build_ctcdecoder(
    vocab_list,
    kenlm_model_path=klm_path,
    alpha=0.5,   # LM weight -- tune against your own validation set
    beta=0.7,    # word insertion bonus -- tune against your own validation set
)

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

transcription = decoder.decode(logits.cpu().numpy()[0], beam_width=100)
print(transcription)

Note on alpha/beta: the values above are starting points, not universal defaults — grid-search them against your own labeled validation set, since optimal weights depend on your specific audio domain.

Intended uses & limitations

• Intended for transcribing spoken Shona audio into cased, punctuated text.
• As a CTC-based model, it assumes single-speaker, forward-only audio and has no mechanism for overlapping speech from multiple speakers.
• Trained on a mix of WAXAL and four community-contributed Shona datasets; acoustic conditions, recording quality, and dialectal coverage reflect that combined pool, not any single controlled source.
• No untouched WAXAL benchmark split remains for this checkpoint (see Training data) — for a clean, comparable WAXAL benchmark score, use the sibling keystats/w2v-bert-2.0-shona-main-best checkpoint instead.

Citation

If you use this model, please cite the training/fine-tuning work and the underlying datasets:

@misc{keystats_wav2vec2bert_shona_bestv2,
  title={w2v-bert-2.0-shona-main-best-2: A Shona ASR model fine-tuned from facebook/w2v-bert-2.0},
  author={keystats},
  year={2026},
  howpublished={\url{https://huggingface.co/keystats/w2v-bert-2.0-shona-main-best-2}}
}

@misc{waxal,
  title={WAXAL: A Multilingual African Speech Dataset},
  author={Google},
  howpublished={\url{https://huggingface.co/datasets/google/WaxalNLP}}
}

@misc{badrex_shona_speech,
  title={shona-speech},
  author={badrex},
  howpublished={\url{https://huggingface.co/datasets/badrex/shona-speech}}
}

@misc{beijuka_shona,
  title={DigitalUmuganda\_AfriVoice\_shona},
  author={Beijuka},
  howpublished={\url{https://huggingface.co/datasets/Beijuka/DigitalUmuganda_AfriVoice_shona}}
}

@misc{shunyalabs_shona,
  title={shona-speech-dataset},
  author={shunyalabs},
  howpublished={\url{https://huggingface.co/datasets/shunyalabs/shona-speech-dataset}}
}

@misc{realtime_speech_shona,
  title={shona2},
  author={realtime-speech},
  howpublished={\url{https://huggingface.co/datasets/realtime-speech/shona2}}
}

@inproceedings{w2vbert2,
  title={Seamless: Multilingual Expressive and Streaming Speech Translation},
  author={Seamless Communication and others},
  year={2023},
  howpublished={\url{https://huggingface.co/facebook/w2v-bert-2.0}}
}