whisper-large-v3-waxal-lin-sna

Whisper large-v3 LoRA adapters for Lingala and Shona (WAXAL)

Two LoRA adapters fine-tuned jointly on Lingala and Shona from the WAXAL corpus. The deployed pipeline this repo supports ensembles between both adapters per utterance (selecting whichever produces the higher beam-search sequence score) — for an exact reproduction of the reported results, load and compare both, not just one.

• swa/ — Stochastic Weight Averaging of the top-3 checkpoints by evaluation WER (steps 4000, 5000, 3000)
• best-checkpoint-4000/ — the single best individual checkpoint (step 4000, WER 0.3234)

LoRA configuration: rank 64, alpha 128, applied to the query/key/value/output attention projections and the two feed-forward sublayers, trained via QLoRA (4-bit NF4) on top of openai/whisper-large-v3.

Usage

Load either adapter individually:

from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel

base = WhisperForConditionalGeneration.from_pretrained(
    "openai/whisper-large-v3", torch_dtype="float16"
)
model = PeftModel.from_pretrained(base, "mrwalker123/whisper-large-v3-waxal-lin-sna", subfolder="swa")
model = model.merge_and_unload()
processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")

Replace subfolder="swa" with subfolder="best-checkpoint-4000" for the second ensemble member. To reproduce the deployed pipeline's exact behavior, generate from both and keep whichever has the higher sequences_scores from beam search per utterance.

Training data

WAXAL corpus (Lingala + Shona train/validation splits). See the accompanying paper for full training procedure, hyperparameters, and evaluation results.