Matcha-TTS

LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).

Measured on device (edge-compat, dp_g2p_matcha_fp16): Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 22.0 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 136 ms p50 (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/matcha-tts__dp_g2p_matcha_fp16/CARD.md

Measured on device (edge-compat, matcha_decoder_fp16): Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 69.7 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 9.55 ms p50 · output differs from CPU (max rel diff 6e+03) (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/matcha-tts__matcha_decoder_fp16/CARD.md

Measured on device (edge-compat, matcha_textenc_fp16): Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 38.7 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 5.99 ms p50 · output matches CPU (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/matcha-tts__matcha_textenc_fp16/CARD.md

Measured on device (edge-compat, matcha_vocoder_fp16): Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 5825 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 104 ms p50 · output differs from CPU (max rel diff 0.22) (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/matcha-tts__matcha_vocoder_fp16/CARD.md

Matcha-TTS — LiteRT (on-device, FFT-free, GPU)

On-device English text-to-speech for Android via LiteRT CompiledModel. This is the FFT-free TTS lane: Matcha-TTS pairs a conditional flow-matching (CFM) acoustic model with a HiFi-GAN time-domain vocoder, so there is no FFT/iSTFT anywhere in the synthesis path. 22.05 kHz, LJSpeech voice.

Try it in your browser: google-ai-edge.github.io/litert-samples/…/matcha-tts — the same four .tflite files below running on LiteRT.js (text encoder + vocoder on WebGPU, decoder on WASM). Nothing to install; inference runs on your machine.

Converted from the official matcha_ljspeech + hifigan_T2_v1 checkpoints with litert-torch, re-authored to be ML-Drift-GPU-clean (per-graph tflite-vs-torch corr 1.000000; end-to-end waveform corr ≥0.99). fp16 weights.

Files

¹ The CFM decoder runs on the CompiledModel CPU delegate. It converts GPU-clean and is correct on CPU, but the Mali ML Drift GPU delegate mis-fuses the decoder's transformer blocks at large activation magnitude (the same block is correct as a standalone GPU graph, corr 0.984, but collapses to corr 0.006 fused — a graph-fusion bug, not a bad op). text encoder + vocoder run on the GPU; the GPU vocoder dominates wall time so the pipeline stays realtime (RTF ~0.8).

Pipeline (host orchestration)

text --G2P(CPU dict+neural)--> phoneme ids
     --host: embed + intersperse + pad-->     text_encoder(GPU) -> mu, logw
     --host: durations + length-regulator-->  mu_y[1,80,T]
     --host: Euler ODE loop (N steps)-->        decoder(CPU) x N -> v
     --host: denormalize-->                     vocoder(GPU)     -> waveform

Fixed shapes (256 phonemes, 512 mel frames ≈ 5.9 s); a runtime float mask makes padded positions a no-op so one compiled graph handles any length.

How to use

Android (Kotlin, LiteRT CompiledModel)

fun load(name: String, acc: Accelerator) =                    // models staged in filesDir
    CompiledModel.create(File(filesDir, name).absolutePath, CompiledModel.Options(acc), null)

val textenc = load("matcha_textenc_fp16.tflite", Accelerator.GPU)
val decoder = load("matcha_decoder_fp16.tflite", Accelerator.CPU)  // Mali mis-fuses this graph on GPU
val vocoder = load("matcha_vocoder_fp16.tflite", Accelerator.GPU)

val teIn = textenc.createInputBuffers(); val teOut = textenc.createOutputBuffers()
teIn[0].writeFloat(emb)     // [1,256,192] host phoneme-embedding lookup (emb.bin), blanks interspersed
teIn[1].writeFloat(tmask)   // [1,1,256]   1 = real phoneme position
textenc.run(teIn, teOut)    // -> mu[1,80,256], logw[1,1,256]
// host: durations ceil(exp(logw))·0.95 -> length-regulate mu -> mu_y[1,80,512]; 10 Euler steps of
// decoder(x, mu_y, t_sin[1,160], ymask[1,1,512]); mel = x·2.116101 − 5.536622 -> vocoder -> wav.
// Full pipeline: the text_to_speech (Matcha-TTS) sample in google-ai-edge/litert-samples.

Python (desktop verification)

import gzip, json, math, numpy as np, soundfile as sf
from ai_edge_litert.interpreter import Interpreter

MAXT, MAXM, LS = 256, 512, 0.95
cfg = json.load(open("config.json"))                     # symbols, mel stats, hop, sample rate
SYM = {s: i for i, s in enumerate(cfg["symbols"])}
DICT = dict(l.rstrip("\n").split("\t", 1) for l in
            gzip.open("g2p_dict.txt.gz", "rt", encoding="utf-8") if "\t" in l)
emb = np.fromfile("emb.bin", "<f4").reshape(178, 192)    # phoneme embedding table

def run(path, *ins):
    it = Interpreter(model_path=path); it.allocate_tensors()
    for d, x in zip(it.get_input_details(), ins): it.set_tensor(d["index"], x.astype(np.float32))
    it.invoke(); return [it.get_tensor(o["index"]) for o in it.get_output_details()]

# text -> espeak-IPA -> symbol ids (dictionary G2P; the neural OOV fallback is skipped here)
ipa = " ".join(DICT[w] for w in "the quick brown fox jumps over the lazy dog".split()) + "."
pids = [SYM[c] for c in ipa if c in SYM]

ids = np.zeros(MAXT, np.int64); ids[1:2 * len(pids):2] = pids   # intersperse blanks (id 0)
tmask = (np.arange(MAXT) < 2 * len(pids) + 1).astype(np.float32)[None, None]
mu, logw = sorted(run("matcha_textenc_fp16.tflite", emb[ids][None], tmask),
                  key=lambda a: -a.shape[1])                    # mu[1,80,256], logw[1,1,256]

w = np.ceil(np.exp(logw[0, 0]) * tmask[0, 0]) * LS              # durations -> length regulator
cum = np.cumsum(w); ylen = int(min(max(cum[-1], 1), MAXM))
mu_y = np.zeros((1, 80, MAXM), np.float32)
mu_y[0, :, :ylen] = mu[0][:, np.searchsorted(cum, np.arange(ylen), "right").clip(max=MAXT - 1)]
ymask = (np.arange(MAXM) < ylen).astype(np.float32)[None, None]

def t_sin(t, half=80):                                          # sinusoidal ODE-time embedding
    e = 1000.0 * t * np.exp(np.arange(half) * -math.log(10000) / (half - 1))
    return np.concatenate([np.sin(e), np.cos(e)]).astype(np.float32)[None]

x = np.zeros((1, 80, MAXM), np.float32)                         # Euler ODE, 10 steps
x[0, :, :ylen] = np.random.randn(80, ylen); N = 10
for k in range(N):
    x += run("matcha_decoder_fp16.tflite", x, mu_y, t_sin(k / N), ymask)[0] / N

mel = np.zeros_like(x); mel[0, :, :ylen] = x[0, :, :ylen] * cfg["mel_std"] + cfg["mel_mean"]
wav = run("matcha_vocoder_fp16.tflite", mel)[0].reshape(-1)[:ylen * cfg["hop"]]
sf.write("out.wav", np.clip(wav, -1, 1), cfg["sample_rate"])

G2P (espeak-free)

Matcha-LJSpeech is trained on espeak en-us IPA, but espeak is GPL. The clean replacement is a 275k-entry espeak-IPA dictionary (from OpenPhonemizer, Clear BSD) as primary + DeepPhonemizer (MIT) on LiteRT CPU for out-of-dictionary words. Output IPA maps 1:1 onto the keithito 178-symbol set.

Sample

See the LiteRT samples/litert/text_to_speech sample (Matcha-TTS) in google-ai-edge/litert-samples for the full Android app and the conversion scripts.

For iOS, samples/litert/text_to_speech/ios runs the same four .tflite files with the LiteRT Swift CompiledModel API.

For the web, samples/web_demos in the same repo has the full browser pipeline on LiteRT.js (G2P, length regulation, the Euler ODE loop, and playback), deployed at the Try-it link above.

Performance

Measured on an Apple M4 Max, CPU/XNNPACK at 8 threads, ai-edge-litert 2.1.6 — median of 15 warm runs per graph, with zero-filled inputs of each graph's declared static shape. Run-to-run spread stayed within 5%.

Summing the graphs as the pipeline calls them — G2P, text encoder, ten decoder steps, vocoder — gives 903 ms of graph time for one 512-frame chunk, which is 5.94 s of audio at 22.05 kHz: RTF 0.15, about 6.6× faster than real time. Host orchestration (dictionary G2P lookup, duration/length regulation, mel denormalization) is not included; it is array bookkeeping, not inference. The vocoder is 77% of the total, so it is the graph to optimize.

For comparison, the card's Android figure is RTF ~0.8 on a Pixel 8a with the text encoder and vocoder on the GPU delegate and the decoder on CPU — a different device and a different backend split.

Android (Pixel 8a)

Android figures use the standard TFLite benchmark_model on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, CPU at 4 threads.

2 of these graphs do not load on the OpenCL delegate at all, so the CPU column is the only Android number for them. The matcha_vocoder_fp16.tflite graph is the exception — it is the one place the GPU pays off here (1177 ms against 5881 ms).

Snapdragon NPU (Hexagon)

• matcha_textenc_fp16.tflite — the NPU is 1.49x faster than the GPU (7.15 ms against 10.69 ms) and loads 5.89x faster (152 ms against 899 ms).

• matcha_decoder_fp16.tflite — the NPU is 1.46x faster than the GPU (9.42 ms against 13.76 ms) and loads 10.04x faster (137 ms against 1373 ms).

• dp_g2p_matcha_fp16.tflite — the GPU delegate declines this graph on the S26: LiteRtException: Failed to compile model. The NPU runs it at 2.06 ms.

• matcha_vocoder_fp16.tflite — did not compile for the Hexagon NPU. The ahead-of-time compile for SM8850 failed, so it never reached the device and has no S26 number on either backend.

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16), LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.70-0.74, where 1.0 is the throttling threshold.

The NPU rows here ran artifacts compiled ahead of time for SM8850 with QAIRT 2.47.0; the GPU rows ran the published files as they are. LiteRT can also compile for the NPU on the device at first load, which is what lets you ship the published file unchanged — that path and the ten runtime libraries it needs are in the NPU recipe, and we did not measure it here. GPU wiring is in the GPU recipe.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT benchmark_model tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

License

Model: MIT (Matcha-TTS / HiFi-GAN). G2P dict: Clear BSD (OpenPhonemizer) + MIT (DeepPhonemizer).