urdu-matcha-tts-female

Matcha Urdu TTS (Female Voice)

A fast, single-speaker Urdu female text-to-speech model, fine-tuned from AhsanTalal/urdu-matcha-tts on a 4.73-hour female voice dataset (2624 utterances).

Based on Matcha-TTS (conditional flow matching, ICASSP 2024).

• Output: 80-band mel spectrogram → waveform (via Vocos)
• Text input: Urdu graphemes
• Speakers: 1 (female)
• Fine-tuning: 700 epochs on female voice data

Audio examples

Example 1

کیا آپ جانتے ہیں کہ میں ایک اردو ٹیکسٹ ٹو سپِیچ ہوں؟ <br><audio controls><source src="https://huggingface.co/AhsanTalal/urdu-matcha-tts-female/resolve/main/examples/example1_kya_ap.wav" type="audio/wav"></audio>

Example 2

سوچتا کوئی اور ہے، ڈھلتا ہے لفظ میں، میں فقط ایک آئینہ ہوں، میرا نام ہے رقم۔ <br><audio controls><source src="https://huggingface.co/AhsanTalal/urdu-matcha-tts-female/resolve/main/examples/example2_sher.wav" type="audio/wav"></audio>

Example 3

اردو ٹیکسٹ ٹو سپِیچ کو استعمال کرنے کے لیے نیچے دیے ہوئے کوڈ کا استعمال کریں۔ <br><audio controls><source src="https://huggingface.co/AhsanTalal/urdu-matcha-tts-female/resolve/main/examples/example3_code_use.wav" type="audio/wav"></audio>

Install the model

pip install --upgrade huggingface_hub
huggingface-cli download AhsanTalal/urdu-matcha-tts-female matcha_tts.onnx vocab.json matcha_urdu_female.ckpt symbols.txt config.yaml --local-dir .

Install the runtime + Vocos vocoder:

pip install onnxruntime numpy soundfile torch torchaudio vocos

The model outputs mel spectrograms — Vocos converts them to audio. Vocos itself is a separate universal model (22.05 kHz), loaded from its own repo below.

How to use (ONNX)

import json
import numpy as np
import torch
import onnxruntime as ort
from vocos import Vocos

# 1. Encode Urdu text (intersperse blank = 0)
stoi = json.load(open("vocab.json", encoding="utf-8"))
space_id = stoi[" "]
text = "ہماری ویب سائٹ پر خوش آمدید"
ids = [stoi.get(ch, space_id) for ch in text]
inter = [0] * (len(ids) * 2 + 1)
for i, v in enumerate(ids):
    inter[i * 2 + 1] = v
x = np.array([inter], dtype=np.int64)

# 2. Run the acoustic model -> 80-band mel
sess = ort.InferenceSession("matcha_tts.onnx", providers=["CPUExecutionProvider"])
scales = np.array([0.667, 1.0], dtype=np.float32)   # [temperature, length_scale]
mel, mel_lengths = sess.run(None, {
    "x": x,
    "x_lengths": np.array([len(inter)], dtype=np.int64),
    "scales": scales,
})

# 3. Install & run Vocos to get audio
vocos = Vocos.from_pretrained("charactr/vocos-mel-22khz")
audio = vocos.decode(torch.from_numpy(mel))

import soundfile as sf
sf.write("out.wav", audio.numpy(), 22050)

How to use (PyTorch checkpoint)

from huggingface_hub import hf_hub_download
import torch
from matcha.models.matcha_tts import MatchaTTS

ckpt = hf_hub_download("AhsanTalal/urdu-matcha-tts-female", "matcha_urdu_female.ckpt")
model = MatchaTTS.load_from_checkpoint(ckpt, map_location="cpu", weights_only=False)
ds = torch.load(ckpt, map_location="cpu", weights_only=False)["hyper_parameters"]["data_statistics"]
model.mel_mean = torch.tensor(ds["mel_mean"])
model.mel_std = torch.tensor(ds["mel_std"])
model.eval()

# Synthesize
out = model.synthesise(x, x_lengths, n_timesteps=10, temperature=0.75, length_scale=0.8)

Fine-tuning details

• Base model: AhsanTalal/urdu-matcha-tts (trained on UrduSpeech)
• Fine-tuning dataset: Female voice Urdu dataset, 2624 utterances, 4.73 hours
• Epochs: 700
• Mel normalisation: mean = -1.9420, std = 2.3366

Training data

Fine-tuned on a female Urdu speech dataset containing 2624 utterances totalling 4.73 hours of audio at 22.05 kHz.

Model details

• Architecture: Matcha-TTS (non-autoregressive, optimal-transport conditional flow matching decoder, euler solver, 10 steps)
• Text processing: grapheme-based Urdu, blank token interspersed between every character (t0 [blank] t1 [blank] ... [blank] tN [blank])
• Acoustic features: 80-band mel, n_fft=1024, hop_length=256, win_length=1024, 22.05 kHz, f_max=8000
• Normalisation: mel mean = -1.9420, std = 2.3366

Limitations

• Single speaker, single style; not designed for voice cloning or code-switching.
• Text outside the vocabulary is mapped to spaces.

References

• Matcha-TTS paper: Mehta et al., "Matcha-TTS: A fast TTS architecture with conditional flow matching", ICASSP 2024 — https://arxiv.org/abs/2309.03199
• Vocos: Lee et al., "Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis", ICLR 2024