phoonnx-omnivoice

phoonnx-omnivoice

ONNX build of OmniVoice by k2-fsa (Xiaomi Corp., authors Han Zhu et al.), packaged for phoonnx.

OmniVoice is a zero-shot text-to-speech model for 600+ languages. It is a masked diffusion language model: a Qwen3-0.6B backbone writes eight streams of Higgs Audio V2 codec tokens, starting from an all-MASK grid and unmasking the most confident slots over 32 steps. Attention is bidirectional, and every step is a full-sequence forward.

Why this build exists

An earlier community export, onnx-community/OmniVoice-Onnx, runs the backbone through the ONNX Runtime contrib operator com.microsoft::GroupQueryAttention. That operator is unconditionally causal. OmniVoice is not a causal model, so the export computes the wrong hidden states. We measured this against upstream PyTorch on a fixed input:

The community llm_decoder reproduces a causal OmniVoice, which is a different model. omnivoice_backbone.onnx here is a fresh export from the PyTorch checkpoint with the bidirectional mask kept intact:

The community Higgs codec graphs are exact and are mirrored here unchanged:

No quantized variant of the backbone is published: the int4 community build was rejected on the agreement test above, and we have not yet produced a quantized export that passes.

Files

Classifier-free guidance runs the backbone twice per step — once over the full prompt and once over the target span alone. Upstream batches both rows behind a [2B,1,S,S] block mask; for a single item that is the same as two forwards of different lengths.

Usage

from phoonnx.model_manager import TTSModelManager

voice = TTSModelManager().load_voice("omnivoice/en")
audio = voice.synthesize(
    "Machine learning models can now speak in hundreds of languages.",
    speaker_reference="reference.wav",
    speaker_reference_text="Transcription of the reference clip.",
)

speaker_reference_text is not optional in practice: OmniVoice joins the reference transcription to the target text into a single prompt string, and cloning is noticeably worse without it.

Verified languages

Word/character error rate on FLEURS, scored with OpenVoiceOS ONNX ASR models and always reported against a floor — the same ASR run over the real human FLEURS recording. See the phoonnx pull request for the full table. Only a sample of the model's 600+ claimed languages has been measured; the rest are untested, not verified.

Licensing and attribution

OmniVoice's code is Apache-2.0. Its released weights are CC-BY-NC because of their training data (Emilia and others), and this repository inherits that: the ONNX graphs are a format conversion of those weights. Non-commercial use only.

Model and method: Zhu Han, Ye Lingxuan, Kang Wei, Yao Zengwei, Guo Liyong, Kuang Fangjun, Han Zhifeng, Zhuang Weiji, Lin Long and Povey Daniel — OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models, arXiv 2604.00688, 2026. The Higgs codec graphs are mirrored from onnx-community/OmniVoice-Onnx.

Do not use this model for voice cloning without the speaker's consent, or for impersonation or fraud.

@article{zhu2026omnivoice,
  title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
  author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and
          Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
  journal={arXiv preprint arXiv:2604.00688},
  year={2026}
}