Downloads · 30 days
6
7% of all-time downloads
mmwanje/waxal-whisper-large-v3-spec_aug_v1
waxal-whisper-large-v3-spec_aug_v1 is a automatic speech recognition model from mmwanje. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as apache-2.0.
Fine-tuned openai/whisper-large-v3 for the Google Waxal ASR challenge (Lingala, Shona, Luganda).
Downloads · 30 days
6
7% of all-time downloads
All-time downloads
84
Public
Parameters
1.5B
6.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.2 GB · 100%
From the Hugging Face model README
Fine-tuned openai/whisper-large-v3 for the Google Waxal ASR challenge (Lingala, Shona, Luganda).
Trained with SpecAugment on precomputed log-mel features from mmwanje/waxal-features-v1 (derived from google/WaxalNLP).
| Base model | openai/whisper-large-v3 |
| Languages | Lingala (lin), Shona (sna), Luganda (lug) |
| Augmentation | SpecAugment (time/freq masking on mel features) |
| Features | Hub dataset mmwanje/waxal-features-v1 (~34k train clips after holdout) |
| Steps | 3000 optimizer steps |
| Effective batch | 16 (per_device_train_batch_size=4, gradient_accumulation_steps=4) |
| Precision | FP16 (bf16 disabled) |
| LR | 1e-5 with 300-step warmup, then linear decay |
| Hardware | 1× NVIDIA RTX A6000 (Thunder Compute) |
| Framework | transformers 4.46.x + custom WaxalSeq2SeqTrainer |
WAXAL_FAST_EVAL=1) to hit the competition deadline; health was monitored via train loss (finite grads, decreasing then plateauing).save_only_model=True / save_total_limit=1 (disk-constrained). Final weights = step 3000.Each Hub push is tagged, e.g.:
run-20260808T0738Z-instance-3oc2pn5t-main-NVIDIA-RTX-A6000
Pin a revision if you need a reproducible load:
from transformers import WhisperForConditionalGeneration, WhisperProcessor
repo = "mmwanje/waxal-whisper-large-v3-spec_aug_v1"
rev = "run-20260808T0738Z-instance-3oc2pn5t-main-NVIDIA-RTX-A6000" # optional
processor = WhisperProcessor.from_pretrained(repo, revision=rev)
model = WhisperForConditionalGeneration.from_pretrained(repo, revision=rev)
Not intended as a general multilingual Whisper replacement outside these languages/domains.
import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor
repo = "mmwanje/waxal-whisper-large-v3-spec_aug_v1"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if device == "cuda" else torch.float32
processor = WhisperProcessor.from_pretrained(repo)
model = WhisperForConditionalGeneration.from_pretrained(repo, torch_dtype=dtype).to(device)
model.eval()
# audio: 16 kHz mono float32 numpy array
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
feats = inputs.input_features.to(device, dtype=dtype)
# Waxal codes → Whisper language tokens used in training:
# lin→lingala, sna→shona, lug→swahili (Whisper has no native Luganda token)
lang = "lingala" # or "shona" / "swahili"
with torch.no_grad():
ids = model.generate(
feats,
language=lang,
task="transcribe",
num_beams=8,
repetition_penalty=1.05,
length_penalty=1.0,
max_new_tokens=225,
)
text = processor.batch_decode(ids, skip_special_tokens=True)[0]
print(text)
Ranked on a Phase‑1 holdout for the small SpecAug sibling; applied as the shortlist for this large model:
beams_8 — num_beams=8, repetition_penalty=1.05, length_penalty=1.0, forced languagebaseline — num_beams=5, same penalties, forced languagerep_1.1 — num_beams=5, repetition_penalty=1.1, forced languageAvoid plain no_cond (no forced language) for SpecAug — it underperformed on the small SpecAug ablations.
Repo helper:
python infer_explore.py --stage apply_phase2_recipes \
--models large_spec_aug --recipes best_small
mmwanje/waxal-features-v1.@misc{waxal-whisper-large-v3-spec-aug-v1,
title = {Waxal Whisper large-v3 SpecAug fine-tune},
author = {mmwanje},
year = {2026},
url = {https://huggingface.co/mmwanje/waxal-whisper-large-v3-spec_aug_v1}
}