nepali-vits-tts

Nepali VITS TTS

Single-speaker Nepali TTS trained from scratch using jaywalnut310/vits on OpenSLR 43.

Try it live: Dragneel/nepali-tts Space

Checkpoints

Use G_100000.pth for inference unless you have a specific reason not to.

Training

• Dataset: OpenSLR 43 — single female studio speaker, 2,002 utterances (~3 hours), 22050 Hz
• Architecture: VITS (jaywalnut310), single-speaker, no phonemizer
• Text representation: Nepali Devanagari graphemes, 71 symbols
• Hardware: AMD MI300X (192 GB VRAM), ROCm 6.2
• Steps: 100k (recommended) and 211k (extended) checkpoints provided
• Config: see nepali_base.json in this repo

Quality observations

This model is honestly limited by dataset size, not architecture.

• At 100k steps, audio is clear and natural for sentences using common vocabulary present in SLR43.
• Continued training past ~150k steps produces no audible improvement on this small (~3 h) corpus — verified with A/B listening across 100k / 200k / 210k checkpoints on the same 3 sentences (see samples/).
• For production-grade Nepali TTS, much larger single-speaker data is needed. Reference: waglesameer5/nepali-tts-v1 uses 65.7 h of single-speaker audio and 1.67 M steps (FastPitch + HiFiGAN).
• A multi-stage pretrain → fine-tune recipe (multi-speaker SLR54 pretrain followed by SLR43 fine-tune) is the documented path to higher quality on this dataset — see project repo's TRAINING_PLAN.md.

Inference

pip install torch soundfile huggingface_hub
python infer_vits1.py "नमस्ते, काठमाडौं एक सुन्दर शहर हो।"

The script automatically clones jaywalnut310/vits, builds the Cython monotonic_align extension, downloads model files, and writes output.wav.

Inference gotcha (critical)

VITS inference requires inserting blank tokens between phonemes via commons.intersperse(t, 0). Without it, audio is garbled regardless of checkpoint quality. The included infer_vits1.py already does this.

Minimal correct inference pattern:

import unicodedata
import commons
from text import text_to_sequence

text = unicodedata.normalize("NFC", text)               # Devanagari NFC
t    = text_to_sequence(text, hps.data.text_cleaners)
t    = commons.intersperse(t, 0)                        # REQUIRED
x    = torch.LongTensor(t).unsqueeze(0)
xl   = torch.LongTensor([len(t)])
audio = net_g.infer(x, xl, noise_scale=0.667,
                    noise_scale_w=0.8, length_scale=1.0)[0][0, 0].cpu().numpy()

Audio samples

Original release samples (single-sentence demos):

A/B/C comparison across checkpoints on identical prompts:

Repository layout

G_100000.pth          # generator (USE THIS)
G_211000.pth          # generator at 211k (no improvement)
D_100000.pth          # discriminator at 100k (for resume)
D_211000.pth          # discriminator at 211k (for resume)
nepali_base.json      # training/inference config
nepali_symbols.py     # 71 Devanagari grapheme symbols
nepali_cleaners.py    # text cleaner (NFC normalize)
infer_vits1.py        # ready-to-run inference script
samples/              # WAV demos

License

MIT. Inherits OpenSLR 43 dataset terms for training data.