Downloads · 30 days
130
1% of all-time downloads
abdouaziiz/wav2vec2-xls-r-300m-wolof-lm
wav2vec2-xls-r-300m-wolof-lm is a automatic speech recognition model from abdouaziiz. Use it when you need speech turned into text. It is set up for transformers. The card lists the license as mit.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
130
1% of all-time downloads
All-time downloads
16.3K
Public
Repo size
3.8 GB
Likes
5
Public
Click a slice to open those files.
.bin1.3 GB · 100%
From the Hugging Face model README
Wolof is a language spoken in Senegal and neighbouring countries, this language is not too well represented, there are few resources in the field of Text en speech In this sense we aim to bring our contribution to this, it is in this sense that enters this repo.
This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m ,with a language model that is fine-tuned with the largest available speech dataset of the ALFFA_PUBLIC
It achieves the following results on the evaluation set:
The duration of the training data is 16.8 hours, which we have divided into 10,000 audio files for the training and 3,339 for the test.
We eval the model at every 1500 step , and log it . and save at every 33340 step
The following hyperparameters were used during training:
| Step | Training Loss | Validation Loss | Wer |
|---|---|---|---|
| 1500 | 2.854200 | 0.642243 | 0.543964 |
| 3000 | 0.599200 | 0.468138 | 0.429549 |
| 4500 | 0.468300 | 0.433436 | 0.405644 |
| 6000 | 0.427000 | 0.384873 | 0.344150 |
| 7500 | 0.377000 | 0.374003 | 0.323892 |
| 9000 | 0.337000 | 0.363674 | 0.306189 |
| 10500 | 0.302400 | 0.349884 | 0 .283908 |
| 12000 | 0.264100 | 0.344104 | 0.277120 |
| 13500 | 0 .254000 | 0.341820 | 0.271316 |
| 15000 | 0.208400 | 0.326502 | 0.260695 |
| 16500 | 0.203500 | 0.326209 | 0.250313 |
| 18000 | 0.159800 | 0.323539 | 0.239851 |
| 19500 | 0.158200 | 0.310694 | 0.230028 |
| 21000 | 0.132800 | 0.338318 | 0.229283 |
| 22500 | 0.112800 | 0.336765 | 0.224145 |
| 24000 | 0.103600 | 0.350208 | 0.227073 |
| 25500 | 0.091400 | 0.353609 | 0.221589 |
| 27000 | 0.084400 | 0.367826 | 0.212565 |
The model can be used directly as follows:
import librosa
import warnings
from transformers import AutoProcessor, AutoModelForCTC
from datasets import Dataset, DatasetDict
from datasets import load_metric
wer_metric = load_metric("wer")
wolof = pd.read_csv('Test.csv') # wolof contains the columns of file , and transcription
wolof = DatasetDict({'test': Dataset.from_pandas(wolof)})
chars_to_ignore_regex = '[\"\?\.\!\-\;\:\(\)\,]'
def remove_special_characters(batch):
batch["transcription"] = re.sub(chars_to_ignore_regex, '', batch["transcription"]).lower() + " "
return batch
wolof = wolof.map(remove_special_characters)
processor = AutoProcessor.from_pretrained("abdouaziiz/wav2vec2-xls-r-300m-wolof-lm")
model = AutoModelForCTC.from_pretrained("abdouaziiz/wav2vec2-xls-r-300m-wolof-lm")
warnings.filterwarnings("ignore")
def speech_file_to_array_fn(batch):
speech_array, sampling_rate = librosa.load(batch["file"], sr = 16000)
batch["speech"] = speech_array.astype('float16')
batch["sampling_rate"] = sampling_rate
batch["target_text"] = batch["transcription"]
return batch
wolof = wolof.map(speech_file_to_array_fn, remove_columns=wolof.column_names["test"], num_proc=1)
def map_to_result(batch):
model.to("cuda")
input_values = processor(
batch["speech"],
sampling_rate=batch["sampling_rate"],
return_tensors="pt"
).input_values.to("cuda")
with torch.no_grad():
logits = model(input_values).logits
pred_ids = torch.argmax(logits, dim=-1)
batch["pred_str"] = processor.batch_decode(pred_ids)[0]
return batch
results = wolof["test"].map(map_to_result)
print("Test WER: {:.3f}".format(wer_metric.compute(predictions=results["pred_str"], references=results["transcription"])))
The results obtained can be improved by using :