pocket-tts-polish-6l

pocket-tts-polish-6l

Polish pocket-tts, distilled from 24 transformer layers down to 6. 109.5 M parameters including the Mimi codec, zero-shot voice cloning from a reference clip, real-time factor 0.21 on a desktop CPU and 0.11 quantised to int8.

The 6-layer model is not a trimmed-down version of the 24-layer one. It is a separate network trained to reproduce the 24-layer model's backbone activations, and on the measurements in this card it comes out ahead of the model it was distilled from — see Why the student beats the teacher.

• 109.5 M parameters: 89.4 M backbone + 20.1 M Mimi codec. float32.
• Word error rate 8.5 % on 1339 utterances across four registers.
• Reads 98.4 % of Polish surnames correctly and 100 % of a 43-word button vocabulary.
• int8 is a load-time flag, not a separate download: load_model(..., quantize=True) gives 1.8× faster inference and 2.7× less memory for at most 0.7 points of accuracy. Numbers below.
• The reference clip changes the output more than anything else about this model. A slow reference and a fast one differ by 22 points on surnames and by a factor of three on word error rate. Details and a selection rule below.

What was measured

1339 utterances, one generation each, no retries, one reference clip, four registers chosen because a single average hides what an application actually hits:

Word error rate over the same set: 8.5 %. Longest silence inside an utterance: median 0.14 s, 90th percentile 0.80 s.

How these were judged, and why two recognisers

Scores here come from two recognisers, not one, and the reason matters more than the numbers.

The first pass is nvidia/stt_pl_fastconformer_hybrid_large_pc (115 M, sherpa-onnx int8, CPU). It is deaf to Polish consonant clusters: it transcribes a correctly spoken Chmielewska as Chilewska, Brzezińska as blizińska, Borkowska as Darkowska. On clips a human confirmed as flawless it scored 47 % where a second recogniser scored 94 %.

So every utterance it marks wrong is re-listened to by nvidia/parakeet-tdt-0.6b-v3, and the verdict is overturned only if the second recogniser hears the target word. The rescue pass was itself checked before being trusted: on deliberately mangled forms (Chilewska, Jakuwska, Marcinak) it repaired 0 of 16, and its false-rescue rate on control words was 0.08 % (2 of 2448). It does not hallucinate the word it is looking for.

Both numbers are given so the difference is visible:

A surname score measured with the first recogniser alone understates this model by 15 points. Any Polish TTS comparison that reports surname accuracy from a single fastconformer pass is reporting a floor, not a result.

The reference clip decides more than the model does

Same weights, same sentences, six different reference clips from six speakers:

(First recogniser only, so these are floors; the comparison between rows is what matters.)

22 points on surnames and a threefold spread in word error rate, from the reference alone. The three slow references take the top three places and the three fast ones the bottom three; correlation between reference tempo and surname accuracy is −0.79 over these six points. Clip duration predicts nothing (+0.06) — all six were 6–9 s.

The mechanism was measured separately: tempo transfers from reference to output with a slope of 0.59, and fast Polish speech loses consonant clusters first. A reference read at roughly 2 words per second of voiced audio is a reasonable target. Six points is a small sample and this is a rule of thumb, not a fitted law.

Practical consequence: check a reference before committing to it. Generate a dozen sentences with it, measure, and pick another if the result is poor. The difference between the best and worst reference here is larger than the difference between this model and the 3× larger one it was distilled from.

Voice cloning

Cosine between WavLM-base-plus-sv x-vectors, 180 generations across the six references above. The absolute number is meaningless without the scale, because two different recordings of one person do not score 1.0 either.

That is 95 % of the way from chance to ceiling. For three of the six speakers the clone scores at or above the ceiling, which means the metric saturates there rather than the clone being more like the speaker than they are themselves.

Speed

12 sentences, about 47 s of generated audio, desktop CPU, no GPU:

Real-time factor is seconds of compute per second of audio; below 1.0 is faster than real time. Phone hardware was not measured. A phone core is commonly 3–5× slower than this desktop, which would put the int8 variant around 0.3–0.6, but that is an extrapolation and not a result.

Quick start

pip install pocket-tts

import soundfile as sf
from pocket_tts import TTSModel

model = TTSModel.load_model(config="hf://shefowl/pocket-tts-polish-6l/config.yaml", temp=0.3)
state = model.get_state_for_audio_prompt("reference.wav")
wav = model.generate_audio(state, "Dzień dobry, nazywam się Katarzyna Brzezińska.")
sf.write("out.wav", wav.numpy().squeeze(), model.sample_rate)

Add quantize=True for dynamic int8. It is applied at load time to the attention and feed-forward layers; the flow head and the Mimi codec stay in float32, so the weights on disk are the same file either way — there is no separate quantised download. What it costs, measured on the same 1339 utterances:

reference.wav ships in this repo and is the clip every number in this card was measured with, with its transcript in reference.txt. It is a held-out clip from CML-TTS (CC-BY-4.0), redistributed here under that licence. samples/ holds five unedited generations from it.

The model has its own Polish sentencepiece tokenizer (4000 tokens, trained on the training transcripts); it ships here and config.yaml points at it. An English tokenizer splits ł ę ś ż into raw UTF-8 bytes and costs 2.1× more tokens per word.

Training

Corpus. 425.1 hours of Polish speech, 176 870 clips, 1248 speakers:

Teacher. kyutai/pocket-tts english_2026-04_24l fine-tuned to Polish: 25 000 steps, batch 64, lr 2e-4 constant with 1000 warm-up steps, text embedding reset because the tokenizer changed. One H100, 2.4 hours wall clock.

Student. Depth distillation, 24 → 6 layers. The student's backbone regresses onto the teacher's backbone activations; the flow head and the EOS head are copied from the teacher and frozen, so single-pass generation stays calibrated. The backbone is seeded from the teacher's bottom three and top three layers rather than randomly. 6000 steps, effective batch 64, lr 8e-4 constant with 500 warm-up steps. One Radeon 7900 XTX, 62 minutes.

6000 steps is where it saturates: at 12 000 steps the surname score moves by one utterance out of 368. Shallower students were trained and are worse — 5 layers 97.3 %, 4 layers 89.1 %, 3 layers 84.5 % — and below 5 layers the real-time factor stops improving anyway, because the codec and the flow head dominate what is left.

Why the student beats the teacher

On the same 368 surname utterances, same reference, same judging: teacher 88.9 %, student 98.4 %. A distilled model is not supposed to beat its teacher, so the gap was tracked down rather than reported.

The teacher sometimes reads the reference clip's text instead of the requested sentence. Asked for Pani Głowacka już wyszła, it says the last words of the reference transcript. This accounts for 8 of the teacher's 41 remaining failures — 2.2 % of all its utterances. The student does it zero times.

The likely reason is what distillation optimises: the student regresses onto the teacher's mean activation, and averaging removes a failure mode that sampling occasionally falls into. Whatever the mechanism, the effect needs capacity — a 4-layer student reproduces the teacher (89.1 % vs 88.9 %) without exceeding it, and only 5 and 6 layers pull ahead.

What is not measured here

• Naturalness. No MOS, no listening study. One informal listening pass by one person compared teacher and student on ten pairs and found them indistinguishable in quality.
• Phone hardware. Every speed number is from a desktop CPU.
• Prosody beyond pause length. Utterances containing a comma pause for 0.40 s at the median against 0.08 s for those without; human conversational speech sits at 0.12–0.23 s and read audiobook speech at 0.55 s. Whether this reads as natural was not assessed.
• Languages other than Polish. The tokenizer and the whole corpus are Polish.
• Long-form text. The longest utterance measured is about 10 seconds.

License and attribution

CC-BY-4.0, inherited from kyutai/pocket-tts. The upstream authors ask that the model not be used for voice impersonation or cloning without explicit and lawful consent, nor to present generated audio as genuine recordings of real people. That request applies to this model too — it clones voices from a few seconds of audio.