swahili-wav2vec2-asr

🇰🇪 Model Card for RareElf/swahili-wav2vec2-asr

This model is a fine-tuned version of eddiegulay/wav2vec2-large-xlsr-mvc-swahili for automatic speech recognition (ASR) in Swahili. It has been trained using the Common Voice 11.0 Swahili dataset.

---

📋 Model Details

Model Description

This model leverages the wav2vec2 architecture from Facebook AI, fine-tuned for Swahili ASR. It maps raw speech waveforms sampled at 16kHz to transcriptions using a CTC (Connectionist Temporal Classification) loss. It supports real-time transcription for voice-based Swahili applications.

• Developed by: Kevin Obote / RareElf
• Funded by: Internal research at Guild Code
• Shared by: RareElf
• Model type: Automatic Speech Recognition (ASR)
• Language(s) (NLP): Swahili (sw)
• License: Apache-2.0
• Finetuned from model: eddiegulay/wav2vec2-large-xlsr-mvc-swahili

Model Sources [optional]

<!-- Provide the basic links for the model. -->

• Repository: https://huggingface.co/RareElf/swahili-wav2vec2-asr
• Paper [optional]: Coming soon
• Demo [optional]: Coming soon on semasasa.ai

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

This model can be used for:

• Transcribing Swahili audio for accessibility, journalism, documentation, education, etc.
• Integration into chatbots or voice agents in Swahili.

Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

• Can be integrated with translation and sentiment analysis pipelines.
• Useful for fine-tuning on domain-specific Swahili data (e.g.education, healthcare, government).

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

• Not suitable for noisy, far-field, or multi-speaker environments without preprocessing.
• Not recommended for use in legal, medical, or high-stakes domains without additional validation.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

• The model may underperform on underrepresented dialects of Swahili.
• Accents, noisy recordings, and overlapping speech may impact accuracy.
• Reflects the linguistic distribution of Common Voice contributors, which may not be representative of all Swahili speakers.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

• Preprocess noisy audio for best results.
• Fine-tune further on targeted domain data for production use.
• Provide user disclaimers about ASR limitations in live deployments.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
import torch
import librosa

model_id = "RareElf/swahili-wav2vec2-asr"

processor = Wav2Vec2Processor.from_pretrained(model_id)
model = Wav2Vec2ForCTC.from_pretrained(model_id)

audio, _ = librosa.load("sample.wav", sr=16000)
inputs = processor(audio, return_tensors="pt", sampling_rate=16000).input_values

with torch.no_grad():
    logits = model(inputs).logits

predicted_ids = torch.argmax(logits, dim=-1)
transcription = processor.batch_decode(predicted_ids)[0]
print(transcription)

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

• Dataset: Common Voice 11.0 – Swahili subset

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

• Fine-tuned using Trainer API from 🤗 Transformers.
• Loss: CTC (Connectionist Temporal Classification)
• Optimizer: AdamW
• Precision: fp16 (mixed precision)

Preprocessing [optional]

• Resampled to 16kHz
• Normalized text
• Removed empty, corrupted, or misaligned samples

Training Hyperparameters

• Training regime:
• Epochs: 10
• Batch Size: 16
• Learning Rate: 3e-4
• Warmup Steps: 500
• Weight Decay: 0.01
• Gradient Accumulation: 2 <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

• Dataset: Held-out subset of Common Voice Swahili

Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

• WER (Word Error Rate)
• BLEU (for translation use-case)
• ROUGE (for paraphrase quality)

Results

Note: Evaluation scores are being finalized with the full test set.

Summary

Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

• Visualized attention maps confirm the model learns phonetic and acoustic patterns relevant to Swahili.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: ***
• Hours used: ~10
• Cloud Provider: Google Cloud
• Compute Region: ****
• Carbon Emitted: ~X gCO2eq (estimation via mlco2)

Technical Specifications [optional]

Model Architecture and Objective

• Architecture: Wav2Vec2 (base) + CTC head
• Objective: Predict character-level transcription from 16kHz audio

Compute Infrastructure

[More Information Needed]

Hardware

** Personal Computer Lenovo ThinkPad T14 Gen 1 (32GB RAM) 1TB SSD

Software

• Python 3.10
• PyTorch 2.x
• Transformers 4.39.x
• Datasets 2.x

Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

• ASR: Automatic Speech Recognition

• WER: Word Error Rate

• CTC: Connectionist Temporal Classification

More Information [optional]

• Common Voice Dataset Card
• Transformers Documentation

Model Card Authors [optional]

• Kevin Obote/ RareElf / Guild Code Team

Model Card Contact

• Email: [email protected]
• GitHub: Kevin Obote