phoonnx-audio8-tts

phoonnx-audio8-tts

ONNX export of Audio8/Audio8-TTS-Preview-0.6b — an 11-language text-to-speech model — packaged for the phoonnx arktts engine.

Upstream ships PyTorch only. These graphs were exported with scripts/conversion/arktts/export_arktts_onnx.py, using the same contract and the same tensor names as the official export of the model's Basque fine-tune, itzune/zortzi-tts-onnx, so graphs from either source are interchangeable.

Languages: Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Spanish.

Usage

from phoonnx.model_manager import TTSModelManager
from phoonnx.voice import TTSVoice

voice = TTSVoice.load(TTSModelManager().get_voice("arktts/audio8-maider/en"))
audio = voice.synthesize("The quick brown fox jumps over the lazy dog.")

Voice ids are arktts/audio8-{maider,antton}/{yue,zh,nl,en,fr,de,it,ja,ko,pl,es}.

Files

slow_ar_fp16.onnx          24-layer backbone, KV-cached          (1.35 GB)
fast_ar_fp16.onnx          4-layer depth transformer              (134 MB)
codec_decoder_fp16.onnx    10 codebooks -> 44.1 kHz waveform      (278 MB)
tokenizer.json             the model's own Qwen2 subword BPE
voices/maider.json         reference codes + transcription
voices/antton.json         reference codes + transcription
runtime_manifest.json      precision and architecture metadata

Voices, and where they come from

Upstream ships no reference voices — Audio8 is a cloning model, and a voice is the codec codes of a short clip rather than an entry in a speaker table. To make it usable as a preset-voice engine, this mirror carries two voices encoded from the reference clips published in itzune/zortzi-tts: voices/maider.wav and voices/antton.wav.

Those clips are from the HiTZ-Aholab Basque TTS dataset (CC BY 4.0). They condition timbre, not language — the model infers the language from the text — so both voices are offered for all eleven languages, and the WER table below is what they actually sound like in each. The clips are Basque, which is not one of Audio8's languages; that is fine for conditioning and is stated here so nobody has to work it out.

The two checkpoints carry a byte-identical codec.pth, so re-encoding those clips with Audio8's own codec reproduces the codes upstream published for Zortzi: codebook 0 matches exactly, and all ten codebooks agree on 99.2 to 99.6 % of frames. The published Zortzi codes are shipped here for that reason — they are the reproducible artifact.

To add your own voice, encode a clip offline with scripts/conversion/arktts/mint_voice.py. This mirror ships the codec decoder only; cloning at synthesis time would need the encoder, which is not exported.

What was verified

Checked against the PyTorch checkpoint (float32) with scripts/conversion/arktts/verify_parity.py, over 24 lockstep greedy decode steps:

The fast AR is noisier here than in the official Zortzi export because the half-precision graph accumulates its RMS norms in half precision rather than single — the explicit float32 round-trip upstream writes cannot survive the fp16 converter. It changes no greedy decision that was not already a tie.

Intelligibility, one sentence per language per voice, transcribed with onnx-community/whisper-large-v3-turbo on CPU:

Cantonese, Chinese, Japanese and Korean are not covered by this run. Whisper is the only model in onnx-asr that claims yue at all, and a Whisper-only Cantonese number would say more about the recogniser than about the voice; when those are measured they should be scored by character (CER), not by word.

CPU real-time factor on twelve cores: 13.1x (Maider), 14.3x (Antton) — slower than the official Zortzi graphs at 8x, because this export carries the tied embedding matrix twice and is not otherwise optimised. This is not a real-time model on CPU.

Licence and attribution

Apache-2.0, from the base model. The model is by Audio8; this repository only exports and repackages it.

The reference clips that carry the two voices are from the HiTZ-Aholab Basque TTS dataset (CC BY 4.0, 10.5281/zenodo.17952596), by way of itzune/zortzi-tts. That licence conditions the voices, so redistributing them or audio generated with them carries the attribution below.

The HiTZ-Aholab speech synthesis dataset was developed with funding from the Ministerio para la Transformación Digital y de la Función Pública and Plan de Recuperación, Transformación y Resiliencia — Funded by EU — NextGenerationEU within the framework of the project ILENIA (ref. 2022/TL22/00215335), and by a grant from the Department of Culture and Language Policy of the Basque Government (IKER-GAITU project).

@dataset{navas_hernaez_2025_17952596,
    author    = {Navas, Eva and Hernaez Rioja, Inmaculada and Saratxaga, Ibon and
                 Sanchez, Jon and García Romillo, Víctor and Flores Ríos, Mariana and
                 Bellanco, Aitor},
    title     = {{HiTZ-Aholab speech synthesis dataset in Basque}},
    month     = dec, year = 2025, publisher = {Zenodo}, version = {1.0},
    doi       = {10.5281/zenodo.17952596},
    url       = {https://doi.org/10.5281/zenodo.17952596}
}

Known limitations

Upstream's, carried over unchanged:

• Numbers are mispronounced — spell them out in the text.
• No text normalization — expand acronyms yourself ("TTS" as "te te ese").
• Sampling is required — greedy decoding never reaches end-of-speech. The engine's defaults (temperature 0.8, top-p 0.95) are upstream's.

Specific to this export:

• Half precision only. The codec decoder is the fp16 graph from itzune/zortzi-tts-onnx, which is valid here because both checkpoints carry the same codec.pth; it is verified above against this model's PyTorch codec.
• No codec encoder, so cloning is an offline step.
• slow_ar_fp16.onnx is about 270 MB larger than it needs to be — the tracer materialises the tied output projection separately from the embedding table.