xtts-v2-nepali

XTTS v2 — Nepali (नेपाली) fine-tune

A fine-tune of Coqui XTTS v2 that adds Nepali (ne) — a language the base model does not ship. XTTS v2 is a zero-shot voice-cloning TTS model: given a few seconds of any voice and a line of text, it speaks that line in that voice. This checkpoint teaches it to do so fluently in Nepali.

• Base model: coqui/XTTS-v2
• Method: full GPT fine-tune (not frozen), restored from the base checkpoint
• Data: 4,140 single-speaker Nepali clips (3,933 train / 207 eval, 95/5)
• Language route: a new ne code registered in config.json, over the existing Devanagari tokenizer
• Output: 24 kHz

Checkpoints

Two checkpoints are provided as self-contained folders. Epoch 10 is recommended — it generalises best; later epochs overfit the single training speaker (eval mel-CE rises while text-CE stays flat).

Each folder is drop-in XTTS format: model.pth, config.json, vocab.json, speakers_xtts.pth.

Usage

from huggingface_hub import snapshot_download
from TTS.tts.configs.xtts_config import XttsConfig
from TTS.tts.models.xtts import Xtts
import torchaudio, torch

d = snapshot_download("Oshara/xtts-v2-nepali", allow_patterns=["epoch-10/*"]) + "/epoch-10"

config = XttsConfig(); config.load_json(f"{d}/config.json")
model = Xtts.init_from_config(config)
model.load_checkpoint(config, checkpoint_path=f"{d}/model.pth",
                      vocab_path=f"{d}/vocab.json",
                      speaker_file_path=f"{d}/speakers_xtts.pth", eval=True)
model.cuda()

out = model.synthesize(
    "नेपालका हिमालहरू संसारभर प्रसिद्ध छन्।",
    config, speaker_wav="reference_voice.wav", language="ne",
    temperature=0.65, repetition_penalty=5.0,
)
torchaudio.save("out.wav", torch.tensor(out["wav"]).unsqueeze(0), 24000)

Tip: for long or numeral-heavy text, normalise first (strip zero-width chars, split on the danda ।, expand Nepali digits to words, insert short inter-sentence silence). This prevents the autoregressive GPT from drifting into noise between sentences.

Evaluation

Evaluated on NepTTS-Bench (205 sentences) plus voice-cloning and prosody metrics. The intelligibility and quality numbers place it among the strongest Nepali systems.

Limitations

• Single-speaker training corpus; timbre diversity comes from XTTS's zero-shot cloning, not the fine-tune.
• Slightly reduced pitch expressiveness vs. natural Nepali speech (flatter intonation).
• Nepali is routed over the existing Devanagari tokenizer; no vocabulary was added.

License

Inherits the Coqui Public Model License (CPML) from the XTTS v2 base model.