vovo

Vovo

An English text-to-speech model built from scratch in Swift with hand-written Metal kernels — its own tensor engine, autograd, optimizer, data pipeline and trainer, no PyTorch anywhere. It was trained on one M2 Max.

Vovo2-21M is the current version: 110 voices, steerable prosody, 20.7 M parameters.

Which one to take

Both live in this repo, at different revisions. Same architecture, same size, different training data — and Vovo2 is not strictly better.

Take Vovo2 to choose a voice, an accent or a pitch range. Take Vovo1.5 if you want one voice and the cleanest speech — 21 M parameters split across 110 voices leave less for any single one, and the word error rate says so.

Use

pip install vovo-mlx
vovo-mlx voices                                       # all 110, with each voice's pitch
vovo-mlx say "The quick brown fox." --speaker p226 -o fox.wav

from vovo_mlx import VovoTTS, SAMPLE_RATE
from vovo_mlx.audio import write_wav

tts = VovoTTS.from_pretrained("franckverrot/vovo")        # Vovo2, 110 voices
write_wav("a.wav", tts.say("Hello.", speaker="p226"), SAMPLE_RATE)
write_wav("b.wav", tts.say("Hello.", speaker="ljspeech", pitch_shift=2), SAMPLE_RATE)

one_voice = VovoTTS.from_pretrained("franckverrot/vovo", revision="vovo1.5-21m")

Delivery is steerable per utterance (pitch_shift in semitones, pitch_scale for range, energy_shift in dB) or per word with SSML — <prosody>, <emphasis>, <break>, <sub> — which is detected automatically:

tts.say('<speak>I said <emphasis level="strong">red</emphasis>,'
        '<break time="700ms"/>not <prosody pitch="+4st">blue</prosody>.</speak>')

Swift is the reference implementation: vovo-core. The Python/MLX port (vovo-mlx) matches it numerically.

Files

Mel convention: 24 kHz, n_fft 1024, hop 256, 100 HTK mel bands, log(clamp(x, 1e-7)).

How it works

Text → normalizer → lexicon G2P → 67 phone symbols → conv prenet and a 6-layer RoPE transformer (d=192) → a per-phone mel prior and log-durations → length regulation → a 6-layer flow-matching DiT decoder (d=384, adaLN-zero) sampled in 16 Euler steps with classifier-free guidance → Vocos. Alignment is learned during training by monotonic alignment search, so no hand-labelled durations. A variance adaptor predicts pitch and energy per phone, which is what the prosody controls steer, and a mel-domain discriminator sharpens the decoder's output where the flow-matching loss would otherwise settle for a blurry average.

How it was trained

Not in one sitting. Each version starts from the previous one's weights, so the cost accumulates:

The first release of this model trained in thirteen minutes; that is long behind us. The interesting part is which hour bought what: the extra voices produced the single biggest gain in pitch accuracy, cutting the error by a third, after further training on one voice alone had stopped helping. More data, not more steps.

Limitations

English only, no streaming. Out-of-vocabulary words are spelled by rule, so unusual names come out odd. Pitch is predicted from text alone, so the model guesses a reading — the prosody controls exist for when it guesses wrong. Intelligible, not studio-clean.

License

MIT (code and weights). Lexicon: ipa-dict (MIT). Vocoder derived from Vocos (MIT). Data: LJSpeech (public domain) and VCTK (CC-BY-4.0).