whisper-large-v3-chichewa

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

whisper-large-v3-chichewa

This model is a fine-tuned version of openai/whisper-large-v3 on a Chichewa dataset. The dataset details will be provided at a later stage. It achieves the following results on the evaluation set:

• Loss: 3.1427
• Wer: 101.3340

Model description

More information needed

Intended uses & limitations

The model is fine-tuned to perfom transcription of Chichewa. There are several versions of this model, please refer to the usage example notebook to see how to find the commit which has model with best WER. Alternatively, you can explore the Files and Versions tab, go to commits and find the commit with best WER (around 61). Also, its worth noting that the model repo doesnt have a tokenizer.json, as a results the model needs to be loaded using WhisperProcessor instead of AutoModel or other modules in Transformer.

Source of Funding for this Work

The dataset used to fine-tune this model, as well as the compute resources, were provided by Opportunity International.
This was part of a project in Malawi aimed at supporting the deployment of an LLM-based chatbot for agriculture, with the capability to handle voice interactions in the local language, Chichewa. A total of 30 hours was collected for this dataset but due to data quality issues, only 25 hours was used. About 30 minutes was also removed to be used as hold-out for further model evaluation.

Training and evaluation data

More information needed

Training procedure

Most of the training for this model involved trying to varying speech dataset sizes (5 hours, 10 hours up to 24 hours). As such, the different model commits represent different data sizes. More details will be added to each model commit.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.45.1
• Pytorch 2.0.1
• Datasets 3.0.1
• Tokenizers 0.20.0