matcha-tts-nepali

Matcha-TTS Nepali

High-quality Nepali text-to-speech using Matcha-TTS (conditional flow matching), trained at 22.05 kHz on the AI4Bharat Rasa dataset, with the Ampixa Nepali G2P frontend and a universal HiFi-GAN vocoder.

• Code / recipe: github.com/ansipd/matcha-tts-nepali (branch nepali)

Features

• Multi-speaker: Female (spk=0) + Male (spk=1)
• 22.05 kHz output
• Nepali G2P: 125 Nepali phone tokens (+ English IPA) → n_vocab=264, via nepa-newa-text-frontend (real_nepali clear-standard profile)
• Universal vocoder: hifigan_univ_v1 (no custom vocoder training needed)

Files

Training Details

• Model: Matcha-TTS (CFM decoder, ~20.9M params), n_spks=2, n_vocab=264, n_feats=80, n_fft=1024, hop=256, f_max=8000
• Sample rate: 22,050 Hz; mel mean=-6.124916, std=2.394981
• Data: 17,840 train / 1,982 val (Rasa Nepali; Female=12,195, Male=7,627 utterances)
• Warm start: VCTK-pretrained Matcha, adapted to n_spks=2 / n_vocab=264
• Optimisation: batch 32 × grad-accum 2 (eff. 64), LR 1e-4 constant, 16-mixed precision, grad-clip 5.0
• Convergence: val loss floor ~2.99 reached by step ~15–25k and flat thereafter; best retained checkpoint is step ~25k (val 2.9905)
• Vocoder for inference: hifigan_univ_v1

Evaluation (Whisper CER on synthesized val prompts)

CER is stable between 20k and 72k, consistent with the val-loss plateau.

Acknowledgements

This model is based on the Matcha-TTS architecture. It was fine-tuned using the AI4Bharat Rasa Nepali dataset. Nepali phonemization was performed using Ampixa's real_nepali G2P frontend (nepa-newa-text-frontend, from the Kala project). We thank the Matcha-TTS authors, AI4Bharat, and Ampixa Labs for making these resources publicly available.

Credits & Attribution

1. Dataset — AI4Bharat Rasa (ai4bharat/Rasa), CC-BY-4.0.

Varadhan, P. S., Sankar, A., Raju, G., & Khapra, M. M. (2024). Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings. Proceedings of INTERSPEECH 2024.

• Rasa is gated on Hugging Face (free access after login + agreeing to share contact info); gating is access-control only and does not change the CC-BY-4.0 license.
• Modifications (CC-BY-4.0 §3(a) "indicate changes"): native 48 kHz audio downsampled to 22.05 kHz, and filtered to 19,822 utterances (Female 12,195 / Male 7,627; train 17,840 / val 1,982). No other audio alteration. These changes are ours, not endorsed by AI4Bharat.

2. Nepali G2P — Ampixa (real_nepali / Kala) (github.com/Ampixa/nepa-newa-text-frontend), CC-BY-SA-4.0 (frontend & associated resources, per the Kala model card); lexicon seed (Google language-resources ne/) CC-BY-4.0. Phonological foundation: Khatiwada (2009).

3. Acoustic model — Matcha-TTS by Shivam Mehta et al. (ICASSP 2024), MIT (Copyright © 2023 Shivam Mehta). The upstream MIT license is retained verbatim in the code repository.

4. Vocoder — HiFi-GAN universal checkpoint (hifigan_univ_v1, 22.05 kHz).

Citation

@inproceedings{mehta2024matcha,
  title     = {Matcha-{TTS}: A fast {TTS} architecture with conditional flow matching},
  author    = {Mehta, Shivam and Tu, Ruibo and Beskow, Jonas and Sz{\'e}kely, {\'E}va and Henter, Gustav Eje},
  booktitle = {Proc. ICASSP},
  year      = {2024}
}

@article{khatiwada2009nepali,
  title   = {Nepali},
  author  = {Khatiwada, R.},
  journal = {Journal of the International Phonetic Association},
  volume  = {39}, number = {3}, pages = {373--380},
  year    = {2009}
}

@misc{ampixa2026kala,
  title  = {Kala: CPU-native Nepali Text-to-Speech with a hand-crafted G2P},
  author = {Ampixa},
  year   = {2026},
  url    = {https://huggingface.co/ampixa/real-nepali-v0.2-kala}
}

@inproceedings{ai4bharat2024rasa,
  author    = {Praveen Srinivasa Varadhan and Ashwin Sankar and Giri Raju and Mitesh M. Khapra},
  title     = {Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings},
  booktitle = {Proceedings of INTERSPEECH 2024},
  year      = {2024}
}

License

• Acoustic model code / recipe: MIT (this repo is a derivative of Matcha-TTS, © 2023 Shivam Mehta; original MIT notice retained in the code repo LICENSE).
• Model weights & samples: CC-BY-4.0 (inherits from the AI4Bharat Rasa training audio).
• Nepali G2P frontend (real_nepali, used at train/inference time): CC-BY-SA-4.0, © Ampixa Labs — see the Kala model card. Attribute Ampixa when redistributing the frontend or its resources.