svara-tts-voiceclone-beta

svara-tts-voiceclone-beta — Voice Cloning + Expressive TTS for Indic Languages

svara-tts-voiceclone-beta is an experimental extension of svara-tts-v1, designed to bring lightweight voice cloning and improved accent preservation to Indic languages. It introduces a simple but effective reference-swap finetuning technique, enabling more stable zero-shot speaker identity across long, expressive utterances.

Built on an Orpheus-style discrete audio token architecture, the model supports 19 languages, expressive cues (<laugh>, <yawn>, <angry>), and low-latency TTS on commodity hardware.

---

At a Glance

• Languages (19): Hindi, Bengali, Marathi, Telugu, Kannada, Bhojpuri, Magahi, Chhattisgarhi, Maithili, Assamese, Bodo, Dogri, Gujarati, Malayalam, Punjabi, Tamil, Nepali, Sanskrit, Indian English.
• Voice Cloning: Improved consistency using reference-swap finetuning, works with short (≈10s) reference audio.
• Expressivity: Emotion tags; non-verbal cues; improved Indic prosody.
• Low-Latency Deployment: Fully compatible with GGUF and vLLM.
• Adaptability: LoRA-ready; easy to specialize for speakers, domains, or dialects.

Demo playback uses the same Space as svara-tts-v1.

---

Prompting (Orpheus-Style)

• Place style/emotion tags at the end: आज शाम को जल्दी मिलते हैं। <neutral>
• Provide reference audio tokens before the target text.
• Use punctuation to control rhythm, pauses, and emphasis.

Zero-shot example:

<BOS>
<reference_audio_tokens_here>
कल शाम को जल्दी मिलते हैं। <neutral>
<SOA>

Speaker IDs remain compatible with svara-tts-v1: Language (Gender).

---

Training Data Summary

svara-tts-voiceclone-beta is enhanced from the multilingual base of svara-tts-v1, trained on:

• SYSPIN, RASA, IndicTTS, SPICOR
• ~2000 hours, ~50 speakers, balanced male/female
• Rich phoneme coverage across 19 Indic languages

The reference-swap augmentation uses multi-utterance samples to improve speaker consistency across Indic phonetic variation.

---

Intended Uses

• Zero-shot voice cloning for Indic voices
• Dialogue systems, IVR, learning apps, accessibility solutions
• Content creation, localization, storytelling
• Research on speech identity, expressivity, and multilingual TTS

Out-of-Scope / Not Intended

• Impersonating private individuals without consent
• Fraud, targeted deception, harassment
• High-risk or safety-critical deployments
• Perfect 1:1 replication of voices (this is a beta research release)

---

Limitations

• Zero-shot cloning is not identical to dedicated finetuning
• Speaker similarity may degrade over long utterances
• Varies by language due to dataset imbalance
• Emotion emphasis may differ across low-resource languages
• Rare names and numbers may require normalization or rewriting

These improve with targeted LoRA finetuning or higher-quality data.

---

Responsible Use

By using this model, you agree to follow applicable laws and ethical guidelines. Synthetic speech should be disclosed when appropriate. Avoid impersonation or harmful use cases.

---

Sources & Links

• Base Model (svara-tts-v1): https://huggingface.co/kenpath/svara-tts-v1
• Demo Space: https://huggingface.co/spaces/kenpath/svara-tts
• Inference Repo: https://github.com/Kenpath/svara-tts-inference
• Indic Text Normalizer: https://github.com/Kenpath/indic-text-normalization

---

🙏 Acknowledgments

Developed by Kenpath Technologies. Special thanks to:

• Canopy Labs — Orpheus (architecture & research release)
• SYSPIN / SPICOR — IISc Bangalore
• AI4Bharat — RASA
• IIT Madras — IndicTTS
• Unsloth (training tools & LoRA insights)
• RunPod (GPU compute credits)

---

License

Apache-2.0

---

Versioning & Changelog

• v0.1.0-beta: Initial release with reference-swap voice cloning