whisper-small-waxal-round2-specaug-v1

Whisper Small Waxal Akan Round 2

Experimental Akan/Twi speech-recognition checkpoint selected at training step 1,200. This model is published for evaluation and community testing; it did not pass every production-promotion gate.

Results

All models below were decoded in one fixed job on all 1,522 rows of google/WaxalNLP, configuration aka_asr, published test split. Decoding used beam size 1, task transcribe, maximum generation length 225, and no forced language token.

Against teckedd/whisper-small-waxal-akan-continuation-v1, the paired 5,000-sample bootstrap candidate-minus-baseline WER interval was -1.39 to -0.06 percentage points, with 98.36% probability of improvement. Round 2 produced 677 better rows, 316 ties, and 529 worse rows.

Training Data and Split

Only google/WaxalNLP/aka_asr was used. The dataset revisions, row IDs, speakers, normalized transcripts, and decoded-audio hashes were frozen before training.

• train: 9,133 rows / 83 speakers
• development: 2,091 rows / 18 speakers
• immutable test: 1,522 rows / 33 speakers
• zero speaker, normalized-transcript, sample-ID, or decoded-audio overlap between partitions
• one train clip outside the 0.4–30 second window removed
• three within-train duplicate clips and one within-development duplicate removed
• one train clip duplicated in development quarantined from train

Audio was decoded as mono float32 at 16 kHz. Text was normalized with Unicode NFC, lowercasing, whitespace collapse, and punctuation removal while preserving Akan letters such as ɛ and ɔ. Training used Whisper log-mel features and training-only SpecAugment.

Training Configuration

• base: openai/whisper-small
• full-model fine-tuning, FP16
• effective batch size: 32
• learning rate: 1e-5
• warmup: 200 steps
• maximum: 2,000 steps
• evaluation/save interval: 200 steps
• selected by unseen-speaker development WER
• selected checkpoint: step 1,200, development WER 32.14%
• training tokenizer prefix: Yoruba proxy, task transcribe
• evaluation: no forced language token

Usage

import torch
from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="teckedd/whisper-small-waxal-round2-specaug-v1",
    device=0 if torch.cuda.is_available() else -1,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
)
asr.model.config.forced_decoder_ids = None
asr.model.generation_config.forced_decoder_ids = None
asr.model.generation_config.language = None
asr.model.generation_config.task = "transcribe"

print(asr("audio.wav", generate_kwargs={"task": "transcribe"})["text"])

Limitations

• One immutable-test output entered a severe repetition loop, repeating ayɛ 62 times.
• Test speaker 4430 regressed from 43.24% to 49.69% WER relative to the continuation checkpoint.
• The model missed the preregistered target of WER below 30.86%.
• Akan spelling, dialect variation, code-switching, names, and noisy recordings remain difficult.
• Do not use this model as the sole basis for medical, emergency, legal, or financial decisions.

The full experiment specification, preprocessing audit, paired evaluation, and failed promotion decision are documented in the Akan Speech Lab.

License

Released under CC BY-SA 4.0 as a conservative choice reflecting the licenses listed for WaxalNLP. Users must review and satisfy the source dataset's attribution and share-alike requirements.