freya-tts-cv-speaker01

FreyaTTS-small (Leyla, extended-coverage fine-tune)

A continued fine-tune of freyavoice/freya-tts (FreyaTTS-small, 183.2M params) on the original Leyla voice, using voice-converted Common Voice tr content to broaden text/phonetic coverage without changing the speaker identity.

This is an independent community fine-tune, not an official Freya release.

What this is

The base model is already voice-locked to Leyla, but its SFT corpus is a relatively narrow, proprietary single-speaker set. This fine-tune keeps the Leyla identity and continues training on a much larger, more diverse text corpus, by:

1. Taking ~19,979 clips (~21.5h) from the highest-clip-count speaker in Mozilla Common Voice tr 26.0 (CC0-1.0) — real, varied Turkish sentences, not synthetic text.
2. Converting every clip's timbre to the Leyla voice with seed-vc (zero-shot voice conversion; diffusion-steps=90, length-adjust=1.0, inference-cfg-rate=0.8), while keeping the original speaker's prosody/content and text transcript. Source audio is silence/click/breath-trimmed adaptively (librosa.effects.trim, top_db=40) before conversion.
3. Full-parameter fine-tuning the original, untouched freyavoice/freya-tts checkpoint on this Leyla-timbre/diverse-content set — not on the raw Common Voice speaker's own voice.

Why this approach

A naive fine-tune on real Common Voice audio would overwrite Leyla's identity with the donor speaker's voice (this is what full-parameter SFT does — see the base model's technical report, section 3.5). Re-voicing the content into Leyla's timbre first lets the model absorb the text/ phonetic diversity of a large real-speech corpus while the identity signal in every training example stays Leyla.

Checkpoint selection and identity check

Candidates were compared by synthesizing fixed sentences at multiple seeds and measuring F0 (pitch) mean/std, both against each other and against the unmodified base model as a reference point:

Step 3500 is within ~2% F0 mean and ~0.7 Hz F0 std of the untouched base model — the identity is essentially preserved. Step 4000 had already started drifting (consistent with the base model's own report finding that continued SFT eventually destabilizes the locked voice), so step 3500 was selected instead of the final checkpoint.

Architecture, inference code, and usage are unchanged from the base model — this repo only replaces the weights.

Usage

Requires the freyavoiceai/FreyaTTS inference code (pip install -r requirements.txt):

from freyatts import FreyaTTS

tts = FreyaTTS.from_pretrained("<this-repo-id>", device="cuda")
wav = tts.synthesize("Merhaba, size nasıl yardımcı olabilirim?")
tts.save_wav(wav, "output.wav")

Data provenance and license

Training text/content is derived from Common Voice tr 26.0, released by Mozilla under CC0-1.0 (public domain dedication). Only the linguistic content and prosody pattern of the donor recordings is used; their timbre is discarded by the voice-conversion step, and every training example's speaker identity is Leyla's, not the Common Voice contributor's. The donor speaker is identified only by an anonymized client_id from the public dataset.

Model weights and code inherit the base model's Apache-2.0 license.

Limitations

Same as the base model: narrowband fidelity ceiling from the AudioVAE2's 16 kHz encode path, digit-dense input requires spoken-form expansion at the text frontend, and long inputs can drift without clause chunking. This fine-tune was not evaluated on the Freya-TR-Eval benchmark; no WER/CER/MOS numbers are claimed here — the checkpoint selection above is an identity-consistency check, not a quality/intelligibility benchmark.

Citation

Base model and architecture: see freyavoice/freya-tts and the FreyaTTS technical report.