chatterbox-cymraeg

chatterbox-cymraeg — Welsh TTS (Chatterbox Multilingual T3, full fine-tune)

Welsh (Cymraeg) text-to-speech with voice cloning, from a full fine-tune of Resemble AI's Chatterbox Multilingual T3 on 100 hours of Common Voice Welsh.

Inference must pass language_id="fi". This is not a mistake — see The [fi] language tag below. Passing "cy" raises ValueError.

Contents

Usage

from chatterbox.mtl_tts import ChatterboxMultilingualTTS
import torchaudio as ta

model = ChatterboxMultilingualTTS.from_local("step13971", device="cuda")
wav = model.generate(
    "Mae'r gwersi yn canolbwyntio ar ysgrifennu i berswadio.",
    language_id="fi",                      # REQUIRED — see below
    audio_prompt_path="reference_speaker.wav",
    temperature=0.8, cfg_weight=0.3, exaggeration=0.5, repetition_penalty=1.5,
)
ta.save("out.wav", wav, model.sr)

The decoding settings above are the ones every number on this card was measured with.

The [fi] language tag

Chatterbox Multilingual supports 23 languages; Welsh is not one of them, and its grapheme tokenizer has no [cy] token (vocabulary of 2,454). The nine [PLACEHOLDER55..63] slots that exist are unreachable, because MTLTokenizer.encode lowercases the tag it constructs, so [PLACEHOLDER55] can never be produced.

So the Finnish slot was reused as a donor tag and re-specialised by the fine-tune. Finnish was picked because it is the slot with published evidence of taking a full new-language fine-tune (Finnish-NLP/Chatterbox-Finnish) and because it shares Welsh's transparent orthography and non-final stress. The model saw only [fi] during training, so the tag now means Welsh to it.

Welsh graphemes are otherwise fully covered: the tokenizer NFKD-normalises before encoding, so ŵ and ŷ decompose into base letter plus combining circumflex and every Welsh grapheme round-trips without [UNK].

Training

Freezing S3Gen is the design decision that matters. It is the component that carries voicing, so keeping it frozen puts the acoustic path out of the optimiser's reach; only the token predictor learns Welsh.

Evaluation

Three frozen evaluation sets, 97 prompts. Identical prompt files, conditioning references (never a prompt's own recording) and per-prompt seeds across every model below. Objective ASR is techiaith/whisper-large-ft-cy-en @ 014eacd47a907794bf093d033488c5edff9cdf58, greedy, language=cy. WER/CER are NFC + casefold + punctuation-stripped, via jiwer.

Voicing (median pitch-band autocorrelation over voiced frames) is reported first and gates the rest: an ASR model will hallucinate fluent Welsh over degenerate audio, so a WER is only meaningful once the audio is known not to be mush. All three sets pass (≥ 0.50).

Against the untouched Chatterbox base under identical decoding, that is a 60% / 47% / 58% relative reduction in mean WER.

Duration and truncation

Caveats

• It speaks faster than the reference humans — about 13% shorter than the human recordings on held-out Common Voice sentences (0.74× on cv16_stage1_test, against 0.845× for the untrained base). The fine-tune compressed delivery toward Common Voice's tighter clip style.
• Long, multi-sentence prompts can get clipped. Of 97 prompts, 8 came out under half the human duration; by WER, 4 are genuinely cut off (the rest are complete but brisk), and all the clipped ones are long FLEURS prompts. Note the untrained base clips 6 of the same 30 FLEURS prompts, so this is a property of the model family and decoding settings, not something the Welsh fine-tune introduced. Split long text into sentences.
• Voicing is level with base, not better: 0.569 against 0.564 on the six-prompt gate set. The fine-tune bought pronunciation, not audio quality — which is the expected result of freezing S3Gen, and the point of doing so.
• WER is one signal. The Welsh ASR has its own dialect and orthographic biases and was itself fine-tuned on Welsh corpora, so it is gentler on Welsh-shaped errors than a listener and harsher on legitimate dialect variation. Listen before trusting.
• Trained on read speech from Common Voice; not validated for expressive, conversational or long-form synthesis.
• Common Voice contributor deletion requests made after the 16.1 snapshot cannot be reflected in a frozen corpus.

Licences

• Weights: MIT, inherited from ResembleAI/chatterbox.
• Training data: CC0-1.0 (techiaith/commonvoice_16_1_cy).
• Generated audio carries Resemble AI's Perth implicit watermark, as base Chatterbox does.

Citation

Base model: Chatterbox, Resemble AI. Data: Common Voice Welsh 16.1, repacked by Techiaith, Bangor University.