XTTS-v2

XTTS v2: Optimized for AMD ROCm

XTTS v2 is a zero-shot, multilingual text-to-speech model that clones a target voice from a few seconds of reference audio and then reads arbitrary text in that voice. This repository packages evaluation/inference for zero-shot multilingual text-to-speech using ONNX Runtime, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs, as well as AMD Ryzen AI NPUs.

This is based on the implementation of XTTS v2 found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the XTTS v2 AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.

---

Task Overview

Task: Zero-shot multilingual text-to-speech / voice cloning

Dataset: 5 fixed English test sentences synthesized from one reference voice, scored against the PyTorch baseline (no labeled corpus — this is a fidelity + quality eval, see Accuracy Pipeline)

Output metrics: RTF, PESQ, STOI, CER, SECS

ONNX Runtime note: CPU runs FP32 only. GPU end-to-end synthesis runs FP32 via ROCMExecutionProvider (the autoregressive GPT decoder needs the dynamic-shape handling that MIGraphX does not provide efficiently); the FP16 GPU speedup is characterized at the vocoder via MIGraphX profiling rather than end-to-end. NPU omits precision entirely (VitisAI auto-quantizes internally).

---

AMD ROCm Optimization

This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs and AMD Ryzen AI NPUs. Key points:

• Exported/tested with ROCm 7.2.
• Validated backends: ONNX Runtime CPU EP, ONNX Runtime ROCM EP (ROCm GPU, with MIGraphX used for GPU profiling), and VitisAI EP (Ryzen AI NPU).
• No code changes required versus the upstream XTTS v2 implementation — only environment/runtime configuration differs.
• CPU fallback path supported for environments without a ROCm-capable GPU.

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the XTTS v2 on GitHub.

---

Model Details

Model Type: Zero-shot multilingual text-to-speech (voice cloning)

Base Model: coqui/XTTS-v2

Model Stats:

• Decomposed into four ONNX sub-models: conditioning encoder, speaker encoder, GPT-2 decoder (run autoregressively with an explicit KV-cache), and a HiFi-GAN vocoder, plus NumPy embedding tables
• GPT decoder: 443M parameters
• Supported languages: en, es, fr, de, it, pt, pl, tr, ru, nl, cs, ar, zh-cn, hu, ko, ja, hi
• Precision tested: FP32 (CPU, GPU); FP16 (GPU vocoder profiling only); Auto/VitisAI-quantized (NPU)

---

Accuracy Pipeline

Some metrics are higher-is-better (PESQ, STOI, SECS) and some are lower-is-better (RTF, CER). Absolute values are modest here by design: TTS sampling is stochastic (temperature 0.75), and the ONNX↔PyTorch fidelity metrics compare two independently sampled utterances of the same text rather than a deterministic reference, so they capture "same speaker/prosody, plausibly the same words," not sample-exact agreement.

Metrics Explained

Accuracy Results

5 English sentences, reference voice = JFK sample — filled from results/validation*/validation_report.json; run make eval-fulldataset-<device> to refresh:

<!-- accuracy-table-start --> <!-- accuracy-table-end -->

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Scripts for the supported runners
• Additional model variants and datasets
• Benchmarking and reproduction instructions