pocket-tts-onnx

pocket-tts-onnx (upstream export)

ONNX graphs for Kyutai's Pocket TTS, exported with the export_onnx.py script that ships with VolgaGerm/PocketTTS.cpp, for use with that C++/ONNX Runtime runtime.

Six languages: english_2026-10, french_24l, german_24l, spanish_24l, italian_24l, portuguese_24l. english_2026-10 (Kyutai's default English model since 2026-10-21, exported with pocket-tts 4.0.1) reads sentence-final numbers again. The older english_2026-04, english_2026-09_2 and english_2026-09_24l folders were removed on 2026-10-29; they stay readable at the earlier commits (for example c1f5d6fb) for anything that still pins one.

Why this exists

The widely used KevinAHM/pocket-tts-onnx export predates PocketTTS.cpp's own exporter. The newer exporter changes the graph, not the weights:

• the Mimi decoder's KV cache is a circular buffer of 250 positions instead of 1000, matching the model's actual attention context — 4× less masked-out work per decode call;
• K and V caches are separate tensors, removing a torch.stack copy per step;
• KV caches are stored in fp16 and bound as both input and output, so ONNX Runtime scatters in place instead of copying the cache every step.

Measured on french_24l, same runtime binary, same reference voice, INT8 backbone + FP32 flow and decoder ("mixed" precision), Apple M-series, median of 5 draws over 3 sentences:

Same intelligibility, ~1.4× faster, ~330 MB less resident memory.

Contents, per language folder

Bundles no longer ship speaker_proj_weight.npy, emb_mean.npy or emb_std.npy (the inputs of a prosody-continuity mode the runtime no longer has): they were removed from every folder on 2026-10-29; earlier commits still carry them.

flow_lm_main.onnx (FP32, 1.2 GB per language) is not published — it is only needed to validate the INT8 quantization, which was done at export time.

Quantizing INT8 Mimi is not free

The INT8 Mimi decoder is included for comparison, not recommended. Against the FP32 decoder on identical latents it scores ~16.5 dB SNR, and the error is chunk-size dependent (ONNX Runtime derives the activation scale from the chunk), correlated with the signal, and concentrated in 2–5 kHz — audible roughness. Kyutai's own runtime never quantizes Mimi or the flow net.

Reproducing

The upstream script is hardcoded to the 6-layer b6369a24 English checkpoint, whose config no longer ships with pocket_tts (upstream issue #12). Two changes were needed:

1. derive FlowLMMainWrapper.NUM_LAYERS from len(flow_lm.transformer.layers) — the *_24l checkpoints have 24 layers, and a hardcoded 6 silently drops 18 layers' worth of state;
2. select the config by language and let pocket_tts resolve the hf://…@rev weights.

Every export was validated against PyTorch, FP32 and INT8, with the script's own run_validation.

Provenance (session 227)

Every bundle.json now records pocket_tts_version (the installed pocket-tts package's release version, or <version>+git<sha12> when installed straight from a git ref via pip's PEP 610 direct_url.json) and export_script_version (a dated stamp bumped whenever tools/pockettts_export/export_onnx.py's export-affecting logic changes) — before this, a downloaded bundle's staleness relative to either the upstream package or this repo's own export tooling was not verifiable without re-running the exporter from scratch. default_temperature is also recorded per language (0.3 for English per Kyutai's own commit d108410, null/absent elsewhere — see the runtime's Bundle::default_temperature for how a bundle without the key falls back unchanged).

Licence and attribution

Weights © Kyutai, released under CC-BY-4.0; these are derived works (ONNX conversions) and carry the same licence. Source model: https://huggingface.co/kyutai/pocket-tts. Export tooling: VolgaGerm/PocketTTS.cpp (MIT).