ema-tts

EMA-TTS

EMA-TTS is a 65M parameter Turkish text to speech model. It reads characters directly and generates 48 kHz audio with a non autoregressive flow matching transformer inside the frozen AudioVAE2 latent space.

On Freya-TR-Eval it reaches 3.0% word error and 0.8% character error, the lowest of any open Turkish system on that benchmark.

License: Apache 2.0.

Technical report: Report.pdf

Results

Comparison figures are as published by the benchmark's authors. Ours are the mean of three full evaluations at different seeds, 3.03 ± 0.24% WER and 0.81 ± 0.06% CER over a range of 2.78 to 3.27, because a single evaluation reports one draw from a stochastic sampler. On average 410 of the 495 sentences come back with no errors at all.

Accuracy improves as sentences get longer, and no clause chunking or retry is used at inference. eval.py reproduces all of this with the benchmark's own harness.

Usage

git clone https://huggingface.co/canberkkkkkk/ema-tts
cd ema-tts
pip install -r requirements.txt

from inference import EmaTTS

ema = EmaTTS.from_pretrained("ckpt", device="cuda")
wav = ema.say("Geçen hafta sonu arkadaşlarımla birlikte şehrin dışına doğru uzun bir yürüyüşe çıktık, hava beklediğimizden çok daha soğuktu ama manzara bütün yorgunluğumuzu unutturacak kadar güzeldi. Yolun yarısında küçük bir kahvaltıcıda mola verdik, sıcak çay içtik ve biraz dinlendikten sonra tekrar yola koyulduk. Akşam eve döndüğümüzde hepimiz çok yorgunduk ama bir sonraki hafta için yeni bir plan yapmaya başlamıştık bile.")
ema.save(wav, "output.wav")

Numbers, dates, times, currency and percentages are expanded to their spoken form before the model reads them, and Turkish case suffixes are recomputed against the reading rather than the figure. say() handles this, so 09:30'da comes out as dokuz buçukta rather than carrying the suffix that was written.

How it works

The model predicts how long each character takes, and those durations place the characters on a timeline. Each frame then attends over the characters near it under learned Gaussian priors, one word either side, so a frame receives a mixture rather than exactly one character.

Two things follow from that. Words connect, because a frame at the end of one word can still see the start of the next and produces the transition. And nothing along the path is rounded, so the acoustic loss reaches the durations and can correct them.

Replacing this with the usual approach, where durations are rounded and each character's representation is repeated that many times, costs 3.78% to 7.28% WER at 100k steps with everything else held fixed. It is the largest single effect we measured.

Details

• Conditional flow matching DiT, 12 layers at width 512, 8 heads, 32 step Euler ODE
• 65M parameters, plus a frozen 94M AudioVAE shared with VoxCPM2
• Text encoder: 4 ConvNeXt blocks, purely convolutional
• Aligner: 4 heads, learned Gaussian widths from 0.18 to 1.2 in word units, one word of context either side
• Latents: 64 dimensions at 25.05 Hz, decoded to 48 kHz
• One speaker, no cloning and no speaker conditioning
• About 1 GB of GPU memory at inference, including the codec

Limitations

Short utterances are the weak case. 8.8% WER at three to six words against 2.2% at ten or more. The training corpus is mostly long form and the model learned a speaking rate that suits it. One and two word utterances are worse still, and the benchmark does not test them.

About six percent of the benchmark is unreachable. Thirty of the 495 sentences fail on every seed. They are mostly proper nouns and rare morphology that the ASR judge mistranscribes even from good audio.

Digits go through the frontend, not the model. A convolutional character encoder cannot count digit groups, so long numbers are expanded to words before the model sees them. This is included and automatic.

Standard orthography only. Slang spellings and lengthened vowels such as çoook or yaaa are out of distribution.

One voice, fixed. No cloning, no speaker control, no style prompts.

Attribution

The AudioVAE is VoxCPM2's, used frozen under Apache 2.0 and downloaded on first use. It decodes latents to 48 kHz and is not trained here.

Output is synthetic speech and should be labelled as such where that is required.

Apache 2.0. Free for commercial use.