whisper-small-nl

whisper-small-nl

This model is a fine-tuned version of qmeeus/whisper-small-nl on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3034
• Wer: 14.5354

Model description

More information needed

Intended uses & limitations

Transcribe files in Dutch:

import soundfile as sf
from transformers import pipeline

whisper_asr = pipeline("automatic-speech-recognition", model="qmeeus/whisper-small-nl", device=0)
whisper_asr.model.config.forced_decoder_ids = whisper_asr.tokenizer.get_decoder_prompt_ids(
    task="transcribe", language="nl"
)

waveform, sr = sf.read(filename)

def iter_chunks(waveform, sampling_rate=16_000, chunk_length=30.):
    assert sampling_rate == 16_000
    n_frames = math.floor(sampling_rate * chunk_length)
    for start in range(0, len(waveform), n_frames):
        end = min(len(waveform), start + n_frames)
        yield waveform[start:end]

for sentence in whisper_asr(iter_chunks(waveform, sr), max_new_tokens=448):
    print(sentence["text"])

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.26.0.dev0
• Pytorch 1.13.0+cu117
• Datasets 2.7.1.dev0
• Tokenizers 0.13.2