Downloads · 30 days
0
walusungungulube/whisper-large-v3-turbo-twi-en
whisper-large-v3-turbo-twi-en is a machine learning model from walusungungulube. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Fine-tunes katrintomanek/whisper-large-v3-turboAkanstandardspeechspecaugment on ghananlpcommunity/kumawood-speech-transcriptions (audio → English subtitle text) for Twi speech → English translation.
Downloads · 30 days
0
Access
Public
Updated Sep 6, 2026
Repo size
276 MB
Likes
0
Public
Click a slice to open those files.
.pt184 MB · 61%
From the Hugging Face model README
Fine-tunes katrintomanek/whisper-large-v3-turbo_Akan_standardspeech_specaugment on
ghananlpcommunity/kumawood-speech-transcriptions (audio → English subtitle text)
for Twi speech → English translation.
Key decisions baked into the pipeline:
yoruba as the
Whisper language token for Twi. We keep language="yoruba" and set
task="translate", so every training label is prefixed with
<|startoftranscript|><|yo|><|translate|><|notimestamps|> and generation is
forced to the same prefix → English output. This matches how the base model
was trained.audio and text (English subtitle) columns are used;
twi_text, film, timing columns are dropped.(film, normalized text), so the most
diverse, duplicate-free set is picked first (unique pass), and only if the
target hours are not reached do repeated lines within the same film get added.r=16, on q_proj/v_proj/k_proj/out_proj) on a single
24 GB A10G (~3.3M trainable params, 0.4% of the model).whisper-twi-en-cache) every SAVE_STEPS (500) via save_strategy="steps".VolumeCommitCallback flushes the volume to
durable storage, so even a hard crash does not lose the last checkpoint.checkpoint-* and resumes with
resume_from_checkpoint, restoring weights, optimizer, scheduler and step
count. Just re-run the same command after a crash.Modal account, CLI installed (pip install modal) and
authenticated. Check your profile:
python -m modal profile list
A Hugging Face token with write access. Create the Modal secret once:
python -m modal secret create huggingface HF_TOKEN=hf_xxxxxxxx
Without this secret training still runs, but the final adapter won't be pushed to the Hub.
Default: all usable non-deduped train segments, shuffled, up to 10 epochs
(epochs=10). Each epoch over the full corpus is roughly 8.5–9 h on one A10G, so a
single attempt covers ~2.5 epochs before the 24 h function timeout; retries
automatically restarts the job, which resumes from the newest checkpoint on the
Volume (up to 10 restarts). Launch detached so it keeps running server-side even
with this machine off:
# non-blocking launch (hidden background client; survives closing the terminal)
.\scripts\launch_detached.ps1
# or interactively / smoke runs:
python -m modal run scripts\train.py
Check status from anywhere:
.\scripts\monitor.ps1
The app builds a cached image, downloads the dataset/model into the persistent
Volume, trains on all non-deduped audio (shuffled), evaluates WER on a held-out
1,000-clip validation sample every SAVE_STEPS, and pushes every checkpoint to
the Hub as its own revision (checkpoint-500, checkpoint-1000, …). When a run
finishes it publishes the best-WER checkpoint to main of
walusungungulube/whisper-large-v3-turbo-twi-en (when HF_TOKEN is present).
Smoke test (tiny run to verify the pipeline end to end):
python -m modal run scripts\train.py --train-hours 0.5 --max-segments 200 --max-steps 2
Reset the volume (wipes checkpoints/final) before a clean retrain:
python -m modal run scripts\clear_volume.py
Edit the constants at the top of scripts\train.py:
| Constant | Default | Meaning |
|---|---|---|
TARGET_HOURS | 1_000_000.0 | Effectively unlimited (train on all audio) |
MAX_SEGMENTS | 500_000 | No practical cap on training segments |
MAX_STEPS | -1 | -1 = epochs × steps-per-epoch (10 epochs) |
EPOCHS | 10 | Target epochs; reached across resumed runs |
PER_DEVICE_BATCH_SIZE | 16 | A10G 24 GB; lower to 8 if OOM |
SAVE_STEPS | 500 | Checkpoint frequency (smaller = faster crash recover) |
LEARNING_RATE | 1e-4 | LoRA learning rate |
LORA_R / LORA_ALPHA | 16 / 16 | LoRA rank/scale |
EVAL_SIZE | 1_000 | Validation clips used for WER |
BASE_MODEL | katrintomanek/... | Checkpoint to start from |
HUB_REPO | walusungungulube/whisper-large-v3-turbo-twi-en | Push destination |
import torch
from transformers import AutoProcessor, WhisperForConditionalGeneration
from peft import PeftModel
base = "katrintomanek/whisper-large-v3-turbo_Akan_standardspeech_specaugment"
model = WhisperForConditionalGeneration.from_pretrained(base, dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, "walusungungulube/whisper-large-v3-turbo-twi-en")
model.eval()
processor = AutoProcessor.from_pretrained("walusungungulube/whisper-large-v3-turbo-twi-en")
forced = processor.get_decoder_prompt_ids(language="yoruba", task="translate")
model.generation_config.forced_decoder_ids = forced
inputs = processor.feature_extractor(audio_array, sampling_rate=16000, return_tensors="pt")
gen = model.generate(inputs.input_features.to(model.device, dtype=torch.bfloat16))
print(processor.batch_decode(gen, skip_special_tokens=True)[0])
text comes from OCR'd subtitles and is a condensed translation, so it may not
cover everything that is said; expect some label noise. WER is reported on the
validation split for reference.