thorsten_vits

ESPnet2 TTS model

jjiang4/thorsten_vits

VITS trained on the Thorsten-Voice Dataset 2022.10 German single male speaker corpus, using the egs2/thorsten/tts1 recipe in espnet. End-to-end, so no separate vocoder is needed.

Trained for 40k steps, 4% of the 1M-step schedule used for the LJSpeech VITS model. Acoustic quality is already past the Tacotron 2 system at this point, but intelligibility is not: alignment is learned by monotonic alignment search, which converges much more slowly than the adversarial waveform objective. For the better word error rate, use jjiang4/thorsten_tts_train_tacotron2_raw_phn_espeak_ng_german with jjiang4/thorsten_hifigan_ft_ljspeech.

Demo

This model was trained on pre-phonemised text (g2p: none), so it expects a space-separated phoneme string, not raw German. Convert the text with the same espeak-ng German frontend the recipe uses:

from espnet2.bin.tts_inference import Text2Speech
from espnet2.text.phoneme_tokenizer import PhonemeTokenizer

g2p = PhonemeTokenizer("espeak_ng_german")
tts = Text2Speech.from_pretrained(
    "jjiang4/thorsten_vits",
    noise_scale=0.333,      # defaults (0.667, 0.8) are worse at this step count
    noise_scale_dur=0.0,
)
text = "im prozess wurden aber nur vierzig fälle thematisiert."
wav = tts(" ".join(g2p.text2tokens(text)))["wav"]

Passing raw text runs but produces garbage: every word maps to the unknown token, and the utterance above comes out 1.0 s long instead of 3.2 s.

Results

100-utterance test set. See the recipe README.

Environments

• espnet version: espnet 202604
• pytorch version: pytorch 2.8.0+cu128
• python version: 3.10.14