roest-v3-chatterbox-500m

Røst-v3-chatterbox-500m

This is a Danish open sourced state-of-the-art Text to Speech (TTS) model. It is trained as part of the CoRal project by the Alexandra Institute.

Looking to improve this model or adapt it to your domain? The finetuning framework used to train this model is fully open-sourced at coral_chatterbox.

Examples

<table> <tr> <th>Settings</th> <th>Mic</th> <th>Nic</th> </tr> <tr> <td colspan="3"><em>Dette er en tekst-til-tale model, trænet til at oplæse dansk. Lyt med og bedøm selv, hvordan den håndterer sprogets mange forskellige lyde og rytmer.</em></td> </tr> <tr> <td><b>temp=0.8, top_p=0.95, c=0.5, min_p=0.05, repetition_pen=2.0</b></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/00_mic_00_t0.8_p0.95_e0.5_c0.5_m0.05_r2.0.wav"></audio></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/00_nic_00_t0.8_p0.95_e0.5_c0.5_m0.05_r2.0.wav"></audio></td> </tr> <tr> <td><b>temp=0.6, top_p=0.95, c=0.5, min_p=0.05, repetition_pen=2.0</b></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/01_mic_00_t0.6_p0.95_e0.5_c0.3_m0.05_r2.0.wav"></audio></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/01_nic_00_t0.6_p0.95_e0.5_c0.3_m0.05_r2.0.wav"></audio></td> </tr> <tr> <td colspan="3"><em>København er Danmarks hovedstad og ligger på øerne Sjælland og Amager, hvor mange turister besøger de smukke kanaler og historiske bygninger.</em></td> </tr> <tr> <td><b>temp=0.8, top_p=0.95, c=0.5, min_p=0.05, repetition_pen=2.0</b></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/00_mic_01_t0.8_p0.95_e0.5_c0.5_m0.05_r2.0.wav"></audio></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/00_nic_01_t0.8_p0.95_e0.5_c0.5_m0.05_r2.0.wav"></audio></td> </tr> <tr> <td><b>temp=0.6, top_p=0.95, c=0.3, min_p=0.05, repetition_pen=2.0</b></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/01_mic_01_t0.6_p0.95_e0.5_c0.3_m0.05_r2.0.wav"></audio></td> <td><audio controls src="https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m/resolve/main/audio_samples/01_nic_01_t0.6_p0.95_e0.5_c0.3_m0.05_r2.0.wav"></audio></td> </tr> </table>

Inference Quickstart

Start by installing the required libraries:

$ pip install chatterbox-tts huggingface_hub torchaudio

To run a simple inference:

import torchaudio as ta
from chatterbox.mtl_tts import ChatterboxMultilingualTTS

REPO_ID = "CoRal-project/roest-v3-chatterbox-500m"
device = "cuda" # Change to cpu if no GPU is available

# Load the model
model_dir = snapshot_download(
        repo_id=REPO_ID,
        token=os.getenv("HF_TOKEN") or True,
        # Optional: Filter to download only what you need
        allow_patterns=["*.safetensors", "*.json", "*.txt", "*.pt", "*.model"],
    )
model = ChatterboxMultilingualTTS.from_local(model_dir, device=device)

text = "Hej, hvordan går det? Jeg er dansk Chatterbox, og taler naturligt dansk."
wav = model.generate(text, language_id="da")
ta.save("test.wav", wav, model.sr)

# If you want to synthesize with a different voice, specify the audio prompt
AUDIO_PROMPT_PATH = "YOUR_FILE.wav"
wav = model.generate(text, language_id="da", audio_prompt_path=AUDIO_PROMPT_PATH)
ta.save("test-2.wav", wav, model.sr)

Model Details

The model is a finetuned variant of Chatterbox Multilingual, one of the leading open-source text-to-speech models. The base Chatterbox model is built on a 0.5B Llama backbone and trained on more than 500,000 hours of high-quality multilingual speech data across 23 languages, including Danish.

We finetuned the model on over 2000 hours of Danish speech to further improve performance and reliability. The model supports zero-shot voice cloning on as little as a 10 seconds audio prompt. It is compatible with the original Chatterbox library, thus being easy to setup and retaining the built-in Watermarked outputs.

The model works very well with the two predefined speakers from the CoRal-tts dataset, Mic and Nic.

Known limitations

• Though the original model supports exaggeration/intensity control, this does not, as the dataset does not support it.
• The model does not handle longer text inputs. Others have implemented streaming for this use case., and we recommend splitting longer texts into sentences.
• The model only supports Danish and English with a heavy Danish accent.

Evaluation

The model was evaluated using Mean Opinion Score (MOS), achieving a score of 4.23 as rated by a panel of 20 native Danish speakers.

The evaluation used a set of 10 samples for two different speakers, Mic and Nic. Samples were generated with temp=0.7, top_p=0.95, top_k=600. The MOS scale was defined as follows:

Training notes

The following hyperparameters were used during training:

Training Data

The model was trained on the following datasets:

Creators and Funders

This model has been trained and the model card written by Daniel Christopher Biørrith at Alexandra Institute.

The CoRal project is funded by the Danish Innovation Fund and consists of the following partners:

• Alexandra Institute
• University of Copenhagen
• Agency for Digital Government
• Alvenir
• Corti

Citation

@misc{roest-v3-chatterbox-500m,
  author    = {Daniel Christopher Biørrith, Dan Saattrup Nielsen, Sif Bernstorff Lehmann, Simon Leminen Madsen and Torben Blach},
  title     = {Røst-v3-chatterbox-500m: A Danish state-of-the-art text-to-speech model},
  year      = {2026},
  url       = {https://huggingface.co/CoRal-project/roest-v3-chatterbox-500m},
}