waxal-llm-asr-1b-milestone-checkpoints

WAXAL LLM-ASR 1B fine-tune (Lingala, Shona, Luganda)

Fine-tune of Meta's omniASR_LLM_1B (Omnilingual ASR) on Lingala, Shona, and Luganda for the WAXAL Hackathon (Zindi).

Full project writeup, training pipeline, and inference script: https://github.com/osinkolu/waxal-hackathon

Checkpoints in this repo

Each step_N/ directory is a full fairseq2 training checkpoint (model + optimizer + trainer + data-reader state) -- resumable for continued training, not just inference. For inference, only step_N/model/pp_00/tp_00/sdp_00.pt (a plain PyTorch state_dict) is needed.

Best: step_124000 (WER ~7.6% on a small in-loop validation sample; step_122500 scored marginally better at the time but wasn't checkpointed).

Important: output format

This model outputs lowercase, unpunctuated "spoken form" text -- omniASR_tokenizer_v1's vocabulary has no entries for uppercase letters or for most punctuation (. , ! ? ; : etc. all map to <unk>; confirmed via direct tokenizer round-trip tests). This isn't a training shortfall, it's what the tokenizer can represent. If you need written-form output, run the transcription through a separate punctuation-restoration tool.

Usage

See inference/inference.py in the GitHub repo linked above -- it downloads a checkpoint from this repo and transcribes audio standalone (Colab-friendly, no training-pod setup needed). Pass nbest=5, compression_window=100, compression_threshold=4.0 for beam search (the inference pipeline's own default of nbest=1 gives noticeably worse results -- less room to recover from an uncertain first token).

Training summary

• Base: omniASR_LLM_1B (full pretrained checkpoint, not a fresh decoder)
• Language conditioning: lang_embeddings_p=0.001 (near-zero, per the paper's Table 22 ablation, matching WAXAL Phase 2's zero-metadata inference constraint)
• 128,000 total training steps across several continuations, driven by real measured throughput and validation trend rather than a pre-committed schedule
• Two tokenizer-vocabulary bugs found and fixed mid-training (capitalization, then punctuation -- see the GitHub writeup for the full story and the exact before/after WER impact of each)