pidgin-wav2vec2-xlsr53

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

pidgin-wav2vec2-xlsr53

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the Nigerian Pidgin dataset. It achieves the following results on the evaluation set:

• Loss: 0.6907
• Wer: 0.3161 (val)

Model description

to be updated

Intended uses & limitations

Intended Uses:

• Best suited for automatic speech recognition (ASR) tasks on Nigerian Pidgin audio, such as speech-to-text conversion and related downstream tasks.
• Academic research on low-resource and creole language ASR.

Known Limitations:

• Performance may degrade with dialectal variation, heavy code-switching, or noisy audio environments.
• Model reflects biases present in the training dataset, which may affect accuracy on underrepresented demographics, phonetic variations or topics.
• May struggle with rare words, numerals, and domain-specific terminology not well represented in the training set.
• Not recommended for high-stakes domains (e.g., legal, medical) without domain-specific retraining/finetuning.

Training and evaluation data

The model was fine-tuned on the Nigerian Pidgin ASR v1.0 dataset, consisting of over 4,200 utterances recorded by 10 native speakers (balanced across gender and age) using the LIG-Aikuma mobile platform. Recordings were collected in controlled environments to ensure high-quality audio. Performance: WER 7.4%(train), 31.6% (validation) / 29.6% (test), exceeding baseline benchmarks like QuartzNet and zero-shot XLSR. This results demonstrate the effectiveness of targeted fine-tuning for low-resource ASR.

Training procedure

We fine-tuned the facebook/wav2vec2-large-xlsr-53 model using the Nigerian Pidgin ASR dataset, following the methodology outlined in the XLSR-53 paper. Training was performed on a single NVIDIA A100 GPU using the Hugging Face transformers library with fp16 mixed precision to accelerate computation and reduce memory usage.

A key modification from the standard setup was unfreezing the feature encoder during fine-tuning. This adjustment yielded improved performance, lowering word error rates (WER) on both validation and test sets compared to the frozen-encoder approach.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-4
• train_batch_size: 4
• eval_batch_size: 4
• seed: 3407
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30
• mixed_precision_training: Native AMP

This configuration balanced training stability, efficiency, and accuracy, allowing the model to adapt effectively to Nigerian Pidgin speech patterns despite the dataset’s limited size

Perfomance Comparision for Frozen Encoder and Unfrozen Encoder:

Training results(Unfrozen Model)

Framework versions

• Transformers 4.37.2
• Pytorch 2.0.1+cu117
• Datasets 2.20.0
• Tokenizers 0.15.2

Citation

@misc{rufai2025endtoendtrainingautomaticspeech, title={Towards End-to-End Training of Automatic Speech Recognition for Nigerian Pidgin}, author={Amina Mardiyyah Rufai and Afolabi Abeeb and Esther Oduntan and Tayo Arulogun and Oluwabukola Adegboro and Daniel Ajisafe}, year={2025}, eprint={2010.11123}, archivePrefix={arXiv}, primaryClass={eess.AS}, url={https://arxiv.org/abs/2010.11123}, }