ASR_french_3

Fine-Tuned Whisper-small Model for French ASR

This model is a fine-tuned version of openai/whisper-small, trained on french version of CV17 dataset

Live demo

Click here (press restart to run the space)

• Then you have two options: Either upload a French audio or record yourself speaking French by clicking on the mic and then the orange dot.

• Hit submit and the model will output the transcription.

Performance and Evaluation

• WER (Word Error Rate): Measures the percentage of words incorrectly predicted.
• CER (Character Error Rate): Measures the percentage of characters incorrectly predicted.

Test Set: CV17(16k samples)

Test Set: MLS (2426 samples)

Usage

import torch

from datasets import load_dataset
from transformers import pipeline

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

# Load pipeline
pipe = pipeline("automatic-speech-recognition", model="nambn0321/ASR_french_3", device=device)


pipe.model.config.forced_decoder_ids = pipe.tokenizer.get_decoder_prompt_ids(language="fr", task="transcribe")

# Load data (this is an example but when you load your own data, make sure to use torchaudio or librosa to load the audio into the dataset)
ds_mcv_test = load_dataset("mozilla-foundation/common_voice_11_0", "fr", split="test", streaming=True)
test_segment = next(iter(ds_mcv_test))
waveform = test_segment["audio"]

# Run
generated_sentences = pipe(waveform, max_new_tokens=225)["text"]  # greedy
# generated_sentences = pipe(waveform, max_new_tokens=225, generate_kwargs={"num_beams": 5})["text"]  # beam search

NOM