nepali-vits2-tts

Nepali VITS2 TTS — 54k Steps (Experimental)

Single-speaker Nepali TTS trained from scratch using p0p4k/vits2_pytorch on OpenSLR 43.

Status: experimental. Trained for only 54k steps as a comparison run against Dragneel/nepali-vits-tts. For best quality on this dataset, prefer the VITS1 model.

Try it live: Dragneel/nepali-tts Space (select "VITS2 (54k steps)")

Training

• Dataset: OpenSLR 43 — single female studio speaker, 2,002 utterances (~3 hours), 22050 Hz
• Architecture: VITS2 (p0p4k port), single-speaker
  • use_transformer_flows: true (pre_conv)
  • use_mel_posterior_encoder: true
  • use_noise_scaled_mas: true
  • use_duration_discriminator: true (dur_disc_1)
  • use_spk_conditioned_encoder: false, n_speakers: 0
• Text representation: Nepali Devanagari graphemes, 71 symbols, no phonemizer
• Hardware: AMD MI300X (192 GB VRAM), ROCm 6.2
• Steps: 54,000
• Config: see nepali_vits2.json in this repo

Inference

pip install torch soundfile huggingface_hub
python infer_vits2.py "नमस्ते, काठमाडौं एक सुन्दर शहर हो।"

The script clones p0p4k/vits2_pytorch, builds the Cython monotonic_align extension, downloads model files, and writes output_vits2.wav.

Inference gotchas

1. Posterior channels = 80: VITS2 with use_mel_posterior_encoder: true uses mel channels (80), NOT filter_length // 2 + 1 (513). The model construction must use 80:

   net_g = SynthesizerTrn(len(symbols), 80,
                          hps.train.segment_size // hps.data.hop_length,
                          **hps.model)
   

2. commons.intersperse(t, 0) is required (same as VITS1) — without it the audio is garbled.

3. Stale repo clones can break inference: this checkpoint was trained against a specific attentions.Encoder.forward signature. If your cloned vits2_pytorch is on an old commit where the inner Encoder does NOT accept g=None, you'll see:

   TypeError: Encoder.forward() got an unexpected keyword argument 'g'
   

   Fix: delete the clone and re-clone fresh from main (the current upstream Encoder.forward(self, x, x_mask, g=None) is compatible). The Space's app.py handles this by re-cloning on every cold start.

Audio samples

Repository layout

G_54000.pth           # generator
config.json           # alt config name (use nepali_vits2.json)
nepali_vits2.json     # training/inference config (canonical)
nepali_symbols.py     # 71 Devanagari grapheme symbols
nepali_cleaners.py    # text cleaner (NFC normalize)
infer_vits2.py        # ready-to-run inference script
samples/              # WAV demos

Why VITS1 is recommended over this VITS2 checkpoint

VITS2 brings architectural improvements (transformer flows, mel posterior encoder, duration discriminator) that should yield better quality given sufficient data and training. On this 3-hour SLR43 corpus at 54k steps, the gains aren't yet apparent vs. the 100k-step VITS1 checkpoint. Both models are bottlenecked by dataset size — see the project repo's TRAINING_PLAN.md for the data-scaling recipe.

License

MIT. Inherits OpenSLR 43 dataset terms for training data.