w2v-bert-2.0-lingala-main-best-last-hope

w2v-bert-2.0-lingala-main-best-last-hope

A Lingala automatic speech recognition (ASR) model, fine-tuned from facebook/w2v-bert-2.0.

This checkpoint follows the exact same training recipe as keystats/w2v-bert-2.0-lingala-main-best, with a single difference: an additional audio duration filter was applied before training, removing clips shorter than 5 seconds and longer than 30 seconds.

Model description

facebook/w2v-bert-2.0 — a large-scale, multilingual self-supervised speech encoder pretrained with a BERT-style masked prediction objective — is used as the backbone, with a from-scratch character-level CTC (Connectionist Temporal Classification) head fine-tuned specifically for Lingala.

Text casing note: training targets were kept in their raw, cased form — same convention as main-best.

Duration filtering note (what differs from main-best): clips with duration below 5 seconds or above 30 seconds were dropped from the training pool prior to fine-tuning, in the hope of reducing the influence of very short, low-context clips and very long clips that disproportionately stress CTC alignment and batching.

Training data

Deduplicated by audio hash and transcription-within-source, then filtered to keep only clips with duration in the 5–30 second range. WAXAL's validation split is the only data used for evaluation, and it was never included in training.

Training procedure

• Base model: facebook/w2v-bert-2.0
• Architecture: Wav2Vec2BertForCTC, add_adapter=True
• Processor: same tokenizer/vocab construction as main-best (character-level, raw/cased text, | word delimiter, [PAD] as CTC blank)
• Sample rate: 16 kHz mono
• Audio duration filter: clips shorter than 5s or longer than 30s removed from the training pool
• Epochs: 4 (with early stopping, patience 5, on validation WER)
• Effective batch size: 32 (per-device batch size 4 × gradient accumulation 8)
• Learning rate: 3e-5, cosine schedule, 10% warmup
• Precision: fp16, gradient checkpointing enabled
• Regularization: attention/hidden/feature-projection dropout 0.05
• Data filtering: CTC-feasibility filtering applied in addition to the 5–30 second duration filter above
• Seed: 42 (deterministic — same seed for Python/NumPy/PyTorch/CUDA)

Evaluation results

Evaluated on the WAXAL Lingala validation split (unfiltered — the duration filter was applied only to training data), greedy decoding vs. greedy + KenLM (keystats/waxal-kenlm-models-best). Adding the KLM gives a consistent, meaningful WER/CER improvement over greedy decoding alone — pair the two for the best results.

How to use

Option 1 — model alone (greedy decoding)

import torch
import librosa
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor

MODEL_ID = "keystats/w2v-bert-2.0-lingala-main-best-last-hope"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]

print(transcription)  # cased, punctuated Lingala text

Option 2 — model + KLM (recommended, higher accuracy)

# pip install pyctcdecode
# pip install https://github.com/kpu/kenlm/archive/master.zip

import torch
import librosa
from huggingface_hub import hf_hub_download
from transformers import Wav2Vec2BertForCTC, Wav2Vec2BertProcessor
from pyctcdecode import build_ctcdecoder

MODEL_ID = "keystats/w2v-bert-2.0-lingala-main-best-last-hope"
KLM_REPO_ID = "keystats/waxal-kenlm-models-best"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

processor = Wav2Vec2BertProcessor.from_pretrained(MODEL_ID)
model = Wav2Vec2BertForCTC.from_pretrained(MODEL_ID).to(DEVICE).eval()

klm_path = hf_hub_download(repo_id=KLM_REPO_ID, repo_type="dataset",
                            filename="lingala/lingala_5gram_correct-best.arpa")

def build_vocab_list(tokenizer, vocab_size):
    vocab_dict = tokenizer.get_vocab()
    vocab_list = [None] * vocab_size
    for tok, idx in sorted(vocab_dict.items(), key=lambda kv: kv[1]):
        if idx < vocab_size:
            vocab_list[idx] = tok
    pad_id = tokenizer.pad_token_id
    if pad_id is not None and pad_id < len(vocab_list):
        vocab_list[pad_id] = ""
    word_delim = getattr(tokenizer, "word_delimiter_token", None)
    if word_delim:
        delim_id = vocab_dict.get(word_delim)
        if delim_id is not None:
            vocab_list[delim_id] = " "
    return vocab_list

vocab_list = build_vocab_list(processor.tokenizer, model.config.vocab_size)
decoder = build_ctcdecoder(
    vocab_list,
    kenlm_model_path=klm_path,
    alpha=0.5,
    beta=0.7,
)

audio_array, sr = librosa.load("path/to/audio.wav", sr=16000, mono=True)
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(input_features=inputs.input_features.to(DEVICE)).logits

transcription = decoder.decode(logits.cpu().numpy()[0], beam_width=100)
print(transcription)

Intended uses & limitations

• Intended for transcribing spoken Lingala audio into cased, punctuated text.
• As a CTC-based model, it assumes single-speaker, forward-only audio and has no mechanism for overlapping speech from multiple speakers.
• Trained on a duration-filtered (5–30s) subset of the main-best pool; performance on utterances outside that duration range (e.g. very short commands or long continuous recordings) is untested and may differ from main-best.

Citation

@misc{keystats_wav2vec2bert_lingala_lasthope,
  title={w2v-bert-2.0-lingala-main-best-last-hope: A Lingala ASR model fine-tuned from facebook/w2v-bert-2.0 on duration-filtered (5-30s) data},
  author={keystats},
  year={2026},
  howpublished={\url{https://huggingface.co/keystats/w2v-bert-2.0-lingala-main-best-last-hope}}
}

@misc{waxal,
  title={WAXAL: A Multilingual African Speech Dataset},
  author={Google},
  howpublished={\url{https://huggingface.co/datasets/google/WaxalNLP}}
}

@misc{kasule_lingala_100hrs,
  title={Lingala\_100hrs},
  author={KasuleTrevor},
  howpublished={\url{https://huggingface.co/datasets/KasuleTrevor/Lingala_100hrs}}
}

@inproceedings{w2vbert2,
  title={Seamless: Multilingual Expressive and Streaming Speech Translation},
  author={Seamless Communication and others},
  year={2023},
  howpublished={\url{https://huggingface.co/facebook/w2v-bert-2.0}}
}