Downloads · 30 days
0
vdub-hindi-dubbing
Bbkblo/vdub-hindi-dubbing
vdub-hindi-dubbing is a text-to-speech model from Bbkblo. Use it when you need text read aloud. The card lists the license as mit.
INPUT = video LINK ya FILE → OUTPUT = Hindi dubbed video.
Access
Public
Updated Aug 22, 2026
Repo size
8.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.3 GB · 49%
At a glance
- Task
- Text-to-Speech
- License
- mit
- Access
- Public
- Created
- Aug 22, 2026
- Updated
- Aug 22, 2026
- SHA
- 33feef27
Model card
From the Hugging Face model README
🎬 vdub — Complete Video Dubbing Pipeline (Hindi)
INPUT = video LINK ya FILE → OUTPUT = Hindi dubbed video.
Chinese (ya kisi bhi) video lo, aur complete dubbing pao: transcript → speaker detection → emotion → translation → voice-cloning TTS (Hindi) → timing/pattern fit → BGM separation → final mux. Saare models isi repo me hain — kahin aur jaane ki zaroorat nahi.
📦 Is repo me SAB KUCH hai
| Cheez | Kahan | Size |
|---|---|---|
| Pipeline code | vdub_full.py (12 stages) | 39 KB |
| Colab/Kaggle runner | notebooks/vdub_full_runner.ipynb | — |
| Chatterbox Hindi TTS (voice clone + emotion) | models/chatterbox_hi/ | 3.2 GB |
| Chatterbox 23-lang pack (optional A/B) | models/mtl23/ | 2.1 GB |
| Kim_Vocal_2 separation (light, default) | models/separation/ | 67 MB |
| MelBandRoformer separation (best quality) | models/separation/ | 913 MB |
| NLLB-200-600M translation (zh→hi) | models/nllb-200-distilled-600M/ | 2.5 GB |
| Pattern transfer (DSP) | scripts/pattern_transfer.py | — |
| System docs | docs/SYSTEM_TREE.md | — |
| Model refill script | fetch_models.sh | — |
Total repo: ~8.8 GB — git clone karte hi sab kuch aa jata hai (LFS).
⚡ Quickstart
A) Notebook (sabse aasaan — Colab/Kaggle)
notebooks/vdub_full_runner.ipynbkholo (Colab: File→Upload notebook; Kaggle: upload/import)- GPU (T4) + Internet ON karo
- Cell 2 me
INPUT_URLyaINPUT_FILEdo - Run All →
vdub_out/dubbed_video.mp4
B) CLI (local GPU / cloud)
git clone https://huggingface.co/Bbkblo/vdub-hindi-dubbing # code + saare models
cd vdub-hindi-dubbing
pip install -e "git+https://github.com/resemble-ai/chatterbox.git" --no-deps
pip install librosa safetensors s3tokenizer transformers huggingface_hub soundfile scipy conformer omegaconf pyloudnorm diffusers ml-collections
pip install "git+https://github.com/resemble-ai/Perth.git@master"
# vdub_full.py ke upar CONFIG me INPUT_URL do, phir:
python3 vdub_full.py
(models agar clone se na mile to: bash fetch_models.sh)
🔄 Pipeline (12 stages, resume-safe)
VIDEO (link/file)
→ 1. fetch (yt-dlp ya file)
→ 2. transcript (SRT ya SenseVoice ASR)
→ 3. clip cutting
→ 4. speaker diarization (campplus clustering)
→ 5. emotion (SenseVoice tags)
→ 6. translation → Hindi (NLLB-200 offline / manual json)
→ 7. TTS (Chatterbox: speaker clone + emotion exaggeration)
→ 8. pattern transfer (original ke pauses/speed/loudness pe fit)
→ 9. separation (Kim_Vocal_2 / MelBandRoformer)
→ 10. mux (SRT timing + instrumental + keep-original ranges + 2 audio tracks)
→ 11. QC (voice similarity report)
→ 12. dubbed_video.mp4 + vdub_output.zip (+ Drive upload optional)
Har stage ka marker banta hai — VM/process mare to wahi se resume.
🎛️ Key config (vdub_full.py ke upar)
| Param | Default | Kya karta hai |
|---|---|---|
INPUT_URL / INPUT_FILE | — | dubbing ka source |
SRT_FILE | "" | apna subtitle (warna ASR) |
TRANSLATE_MODE | nllb | nllb (offline) / manual (json) |
SPEAKER_MAP | "" | {"Speaker 1": "Hero"} names |
SONG_RANGES | "" | [[s,e],...] — yahan original audio rahega |
SEP_MODE | kim | kim (67MB) / melband (913MB) |
DRIVE_UPLOAD | False | output Drive me bhi bhejo |
TTS knobs: exaggeration (emotion: ANGRY 1.4 / HAPPY 1.1 / SAD 0.4 / NEUTRAL 0.5),
cfg_weight 0.3 (cross-lingual accent kam), temperature 0.8.
🤖 Models — license table
| Model | License | Use |
|---|---|---|
| Chatterbox t3_hi + S3Gen + ve (ResembleAI) | MIT | TTS core |
| Chatterbox t3_mtl23ls (23-lang) | MIT | optional A/B |
| Kim_Vocal_2 (UVR5) | MIT | separation default |
| MelBandRoformer (FunAudioLLM) | Apache-2.0 | separation best |
| NLLB-200-distilled-600M (Meta) | CC-BY-NC-4.0 | translation (non-commercial) |
| SenseVoice/campplus (funasr) | auto-download | ASR/emotion/speaker QC |
⚖️ Honest notes
- Chatterbox output me PerTh watermark embed hota hai
- Cloned voice use karne se pehle awaaz wale insaan ki permission lein
- NLLB translation CC-BY-NC hai — commercial use ke liye apna translation API lagao
- funasr ke chhote models (SenseVoice/campplus) pehli baar auto-download hote hain
👥 Credits
- Voice cloning + Hindi TTS: Resemble AI (Chatterbox, MIT)
- Separation (best): FunAudioLLM (Fun-CineForge, Apache-2.0)
- Translation: Meta (NLLB-200)
- Pipeline + pattern transfer: vdub project (Azim Khorajiya)
- Task
- Text-to-Speech
- License
- mit
- Languages
- hi, zh
- Created
- Aug 22, 2026
- Updated
- Aug 22, 2026