roest-v3-wav2vec2-315m

Røst-v3-wav2vec2-315m

This is a Danish state-of-the-art speech recognition model, trained as part of the CoRal project by the Alexandra Institute.

This repository contains a Wav2vec2-XLSR-300M model trained on the CoRal-v3 dataset. The CoRal-v3 dataset includes a rich variety of Danish conversational and read-aloud data, distributed across diverse age groups, genders, and dialects. The model is designed for automatic speech recognition (ASR).

Quick Start

Start by installing the required libraries:

$ pip install transformers

Next you can use the model using the transformers Python package as follows:

>>> from transformers import pipeline
>>> audio = get_audio()  # 16kHz raw audio array
>>> transcriber = pipeline(model="CoRal-project/roest-v3-wav2vec2-315m")
>>> transcriber(audio)
{'text': 'your transcription'}

Model Details

Wav2vec2 is a state-of-the-art model architecture for speech recognition, leveraging self-supervised learning from raw audio data. The pre-trained Wav2vec2-XLS-R has been fine-tuned for automatic speech recognition with the CoRal-v3 dataset dataset to enhance its performance in recognising Danish speech with consideration to different dialects. The model was trained using the CoRal model training framework by running:

uv run accelerate launch \
  --use-deepspeed \
  --zero-stage 2 \
  src/scripts/finetune_asr_model.py \
  model=wav2vec2-small \
  per_device_batch_size=64 \
  max_steps=100000

Note that the dataset, and thus also this model, is licensed under a custom license, adapted from OpenRAIL-M, which allows commercial use with few restrictions (speech synthesis and biometric identification) - see license.

---

Evaluation

The model was evaluated using Character Error Rate (CER), which is the percentage of characters incorrectly transcribed.

Conversational CoRal Performance

<img src="https://cdn-uploads.huggingface.co/production/uploads/60d368a613f774189902f555/EvCjm3ui4t9F0Wq1c_pms.png" width="1000px">

Read-aloud CoRal Performance

<img src="https://cdn-uploads.huggingface.co/production/uploads/60d368a613f774189902f555/duGymBAttMwxAEQ738zEo.png" width="1000px">

Creators and Funders

This model has been trained and the model card written by Dan Saattrup Smart at the Alexandra Institute.

The CoRal project is funded by the Danish Innovation Fund and consists of the following partners:

• Alexandra Institute
• University of Copenhagen
• Agency for Digital Government
• Alvenir
• Corti