bhojpuri-f5-tts

<p align="center"> <img src="logo.gif" width="96" alt="Bhojpuri-F5-TTS mascot, speaking"> </p> <h1 align="center">Bhojpuri-F5-TTS</h1> <p align="center">A LoRA fine-tune of <a href="https://huggingface.co/ai4bharat/IndicF5">IndicF5</a> for improved Bhojpuri pronunciation and prosody, trained on the SYSPIN Bhojpuri corpus.</p>

What this is

A rank-32 LoRA adapter (10.1M / 347M trainable params, 2.9%) fine-tuned on top of AI4Bharat's IndicF5 (F5-TTS architecture) using 9.4 hours of the SYSPIN Bhojpuri corpus (2 studio speakers, male + female). Zero-shot voice cloning: give it a short reference clip + its transcript, and target text, and it synthesizes that text in the reference voice/style.

This is a personal, self-funded learning project — not a production system. Read the caveats below before using it for anything beyond experimentation.

Does it actually help?

Measured against stock IndicF5 on a 32-sentence held-out diagnostic set built specifically to probe Bhojpuri-vs-Hindi pronunciation contrasts (व→ब, श/ष→स, ण→न, avagraha, verb endings, retained final vowels), using pitch-contour correlation against real recordings of the same sentences (DTW-aligned):

Improved on all 8 pronunciation categories tested, biggest gains on avagraha (0.628→0.785) and retroflex_n (0.549→0.696).

Left: training/validation loss over the 8,000-update run — note it plateaus hard after ~update 2,500-3,000; most of the gain happens early. Right: per-category pitch-correlation vs. stock on the 32-sentence diagnostic set.

Read the numbers honestly, not as a guarantee:

• These are pitch/spectral proxies for melody and prosody, not a measure of correct pronunciation or intelligibility — no native-speaker listening evaluation has been done yet.
• 32 diagnostic sentences, one seed, one real reference recording per sentence.
• The training slice (9.4h) is on the smaller side for this kind of fine-tune — treat this as a promising checkpoint, not a finished model.
• "Duration drift" in this project's own diagnostics is identical across all models (F5 sets output length from text) and isn't a model quality signal.

Usage

This is a LoRA adapter on IndicF5's custom F5-TTS architecture (not a transformers model), so it's loaded through this project's own code rather than AutoModel. Verified working end-to-end:

git clone https://github.com/gentleman101/bhojpuri-f5-tts.git
cd bhojpuri-f5-tts
./scripts/bootstrap.sh                    # sets up the venv, downloads IndicF5 base weights
python scripts/pull_run.py lora_10h_r32   # downloads this checkpoint from this repo

python scripts/infer.py \
  --ref-audio your_reference.wav --ref-text "transcript of your reference clip" \
  --text "अरे भाई, ई त बहुते बढ़िया बा!" \
  --adapter runs/lora_10h_r32/checkpoints/step_0008000 \
  --out out.wav

--ref-audio is a 3-10s clip of any voice you want to clone (any language/speaker — zero-shot), with --ref-text as its exact transcript. Full training/inference code, configs, and the diagnostic set: github.com/gentleman101/bhojpuri-f5-tts

Data & attribution

Fine-tuned on the SYSPIN_S1.0 Bhojpuri corpus, © Indian Institute of Science (IISc) Bengaluru, SPIRE Lab, released under CC-BY-4.0. Please cite:

Abhayjeet et al., "SYSPIN_S1.0 Corpus — A TTS Corpus of 900+ hours in nine Indian Languages", 2025.

Base model: ai4bharat/IndicF5 (MIT license).

License & ethical use

• Code (training/inference scripts, this repo's tooling): MIT, see the GitHub repo.
• Model weights: derived from CC-BY-4.0 data; released under CC-BY-4.0 with the attribution above.
• This model can reproduce the voice characteristics of the two SYSPIN speakers. It is released for research/educational use consistent with SYSPIN's stated purpose (building open TTS technology for underserved Indian languages). Do not use it to impersonate these or any other individuals, to create deceptive or fraudulent audio, or to imply endorsement by the original speakers. Audio generated with this model is synthetic — disclose it as such wherever it's used.

Training details

• Base: IndicF5 (F5-TTS arch, ~330M params), LoRA rank 32, alpha 32, targeting attention + feedforward projections.
• Data: syspin_10h split, 5,462 training clips / 100 validation clips.
• 8,000 updates, batch ≈38,400 frames/update, AdamW, peak LR 1e-4 (800-update linear warmup, linear decay to 0), bf16, EMA decay 0.999.
• Hardware: single A100 40GB, torch.compile enabled (~1.8x speedup over eager).
• Final validation loss: 0.6173 (EMA weights).