pocket-tts-farsi

Pocket TTS — Farsi (Persian)

pocket-tts-farsi-v2 supersedes this model

v2 is trained on 973 hours from 2,978 speakers against this model's 497 hours and 1,100. On 300 held-out Common Voice pairs it is more stable and clones voices better:

	v1 (this model)	v2
mean WER	2.048	0.582
median per-item WER	0.333	0.333
runaway generations	25 / 300	0–1 / 300
speaker similarity	0.764	0.859

Those numbers cannot see the ezafe, which Persian does not write, so no transcription-based metric contains it. A blind listening test settles that separately: on 34 sentences scored by a native listener with the models hidden, v2 won on ezafe 21 to 1, on first-word accuracy 22 to 0, on phrasing 24 to 1 and on naturalness 26 to 3.

The other real difference: v2 takes romanised phonemes, so it needs a grapheme-to-phoneme step in front. This model reads Persian script directly.

Use v2 unless you specifically want Persian script input. That is the one thing this model still does better: it needs no G2P stage, no phoneme notation, and no extra dependency. Everything else — accuracy, stability, voice cloning, and prosody by ear — goes to v2.

This model stays published, unchanged, for anyone already depending on it.

Code, training pipeline and full results: github.com/mallahyari/pocket-tts — see training/farsi/RESULTS.md for the measured numbers behind this model, and training/farsi/RUNBOOK.md for how it was built, step by step. v2's report is training/farsi/v2/RESULTS.md.

Usage

The easy way: farsi_tts.py

This repo includes a self-contained script with the settings below already built in, safe chunking for long text, automatic recovery when a generation fails to stop, and the exact Persian text normalization the model was trained on. Download farsi_tts.py and normalize_fa.py (they must stay in the same folder) and run:

uv run farsi_tts.py --text "سلام، حال شما چطور است؟"
uv run farsi_tts.py --text-file article.txt --out article.wav
uv run farsi_tts.py --list-voices

uv run builds an isolated environment on first use — no manual pip install. See farsi_tts.py --help for every option (voice selection, chunk size, pause lengths).

Directly with pocket-tts

# example_voice.wav is included in this repo
uvx pocket-tts generate --config hf://mehdi-hf/pocket-tts-farsi/farsi.yaml \
    --voice example_voice.wav \
    --text "سلام، حال شما چطور است؟" \
    --temperature 0.3 --eos-threshold -2 --frames-after-eos 0

Prompts must be 16-bit WAV unless you add soundfile: uvx --with soundfile pocket-tts generate ... for FLAC, MP3 and the rest.

From Python:

from pocket_tts import TTSModel

model = TTSModel.load_model(config="hf://mehdi-hf/pocket-tts-farsi/farsi.yaml", temp=0.3)
state = model.get_state_for_audio_prompt("your_voice.wav")
audio = model.generate_audio(state, "سلام، حال شما چطور است؟", frames_after_eos=0)

Recommended settings. Classifier-free guidance is baked in by distillation — there is no --cfg flag on this model's generate command, and none is needed. --frames-after-eos 0 trims a trailing inhale the model learned from its training data. --temperature 0.3 and --eos-threshold -2 were chosen by listening. Text longer than one short sentence should be chunked — see farsi_tts.py above, or split it yourself at sentence/clause punctuation into pieces under ~18 tokens; the raw CLI has no chunking of its own and can run past its length cap on long, unpunctuated input.

Voice prompts

This is where the model's main limitation shows. Speaker similarity is 0.948 on voices resembling its training data but 0.728 on unfamiliar speakers, and on an unfamiliar voice its stability degrades as generation runs longer — it can fail to stop, producing a capped, garbled clip.

What works:

• Start from example_voice.wav (included). It is the closest match to the training distribution and gives the best results by a clear margin.
• Clean audio, one speaker, no music. Noisy prompts measurably increase repetition failures.
• Keep prompts short — around 2-5 seconds. Training capped voice prompts at 5 s, so longer ones are out of distribution.
• Chunk text into ~18-token pieces with an unfamiliar voice. Longer chunks are where "maximum generation length reached without EOS" appears: measured on held-out speakers, chunks of 21+ tokens ran past EOS deterministically (no seed recovered them), while 9-16 token chunks were clean.

Persian voices similar in character to broadcast narration clone best. Strongly-accented, very high-pitched, and children's voices are under-served by the training corpus.

Long text must be chunked. The average training utterance is 3.8 seconds (~11 tokens), so paragraphs are out of distribution. Split at Persian punctuation into chunks under ~18 tokens and join them with ~0.15 s of silence. farsi_tts.py does this for you; the splitter it uses is also in the training repo as training/farsi/synthesize.py.

Results

Scored with whisper-large-v3 (language="fa"). floor is that ASR's own WER on the real recordings — the measurement floor. Persian ASR is far from perfect, so compare against the floor, not against zero.

Clean set (hand-verified transcripts, studio narrator), 100 items:

Held-out speakers (unseen voices, subtitle transcripts), 50 items. loops counts items that collapsed into repetition:

Whisper disagrees with those subtitle transcripts 48% of the time on the real recordings, so roughly half of that apparent error is not the model's.

Training data

All CC0, ~497 hours after filtering:

Word alignments from m3hrdadfi/wav2vec2-large-xlsr-persian-v3 (99.96% of utterances aligned). Text normalized to a single Persian spelling — Arabic letter forms folded to Persian, harakat stripped, ZWNJ preserved, all digit systems converted to words. Tokenizer: sentencepiece BPE, vocab 4000.

Teacher: 24 layers, 316M params, 400k steps at effective batch 64. Student: 6 layers, 200k steps of depth distillation with guidance baked in. Both on 8×H100.

Limitations

Ezafe. Persian does not write the linking -e between a noun and its modifier, so the model must infer it from context and often does not: حملات برون‌مرزی gets a short pause where the ezafe belongs instead of being read as hamalāt-e borun-marzi. Writing the kasre in the input does not help — the text normalizer strips harakat. Fixing this properly requires a phoneme front-end with ezafe restoration.

Repetition loops. The model can fail to emit end-of-speech and run to its length cap, filling the tail with repetition — roughly 18% of held-out items with noisy voice prompts, and 9% on a 300-item held-out set with clean ones.

farsi_tts.py mitigates this rather than fixing it: it detects a generation that hit the cap, retries once (sampling is stochastic, so marginal cases recover), and otherwise splits the text and regenerates the halves, which resolved every deterministically-stuck case tested. Using the raw CLI or the Python API directly, you get no such recovery — keep chunks short. A real fix needs retraining on longer utterances.

Short-form bias. Trained on 3.8-second utterances on average; chunk long text rather than sending it whole.

Domain. Mostly film dialogue, podcasts and magazine narration. Under-served: regional accents, children's voices, and (from Common Voice's absence here) crowd-sourced read speech.

Licence and attribution

MIT, matching the pocket-tts code. All training data is CC0. If you use this, credit is appreciated but not required.

Built with pocket-tts by Kyutai; the model architecture follows the CALM paper.