SiangTTS-VoxCPM2-Thai-LoRA

SiangTTS — Thai Voice-Cloning TTS (VoxCPM2 LoRA)

SiangTTS (เสียง = voice) is a LoRA adapter for openbmb/VoxCPM2 that gives it clear, natural Thai speech with zero-shot voice cloning, trained on a single RTX 3090 (24 GB).

It's a parameter-efficient (LoRA) fine-tune of VoxCPM2 — small enough to train on one consumer GPU while keeping the base model's voice-design and cloning abilities. The Thai-adaptation approach is inspired by JaiTTS (a separate, closed-source Thai VoxCPM model by others); SiangTTS is an independent open reproduction at LoRA scale and is not affiliated with it.

🔊 Listen / compare (ref vs ground-truth vs base vs SiangTTS): https://dubbing-ai.github.io/VoxCPM-thai/ · Code: https://github.com/dubbing-ai/VoxCPM-thai

Results

Measured with Typhoon-Whisper-Large-v3 (Thai ASR; CER) and WavLM x-vectors (speaker SIM) on small eval sets, so numbers are directional. CER is an upper bound on error — the ASR judge itself mis-recognises some rare/archaic Thai words the model pronounces correctly.

VoxCPM2's base is already a capable Thai speaker (it reads numerals and handles long-form). SiangTTS clones at essentially the real same-speaker similarity ceiling and is as intelligible as the original recordings, with much lower CER than the base. The GT column is the real recording, included as the reference.

(CER ≤ GT reflects clean, ASR-friendly synthesis plus the ASR judge's own floor; SIM ≈ GT means cloning is about as close as two real recordings of one speaker.) Separately, short-form Thai CER 5.7%→3.8% and long-form 2.7%→1.6% (small 5 / 2 -prompt sets — directional).

Trained 2 epochs over ~205 h: Common Voice Thai (diverse speakers) + porjai_central (studio-clean) + a LibriTTS-R English slice (retains English & code-switching). Audio encoded at 16 kHz, generated at 48 kHz (VoxCPM2 design).

Usage

from voxcpm import VoxCPM
from voxcpm.model.voxcpm2 import LoRAConfig
import json

cfg = json.load(open("lora_config.json"))["lora_config"]
model = VoxCPM.from_pretrained(
    "openbmb/VoxCPM2",
    lora_config=LoRAConfig(**cfg),
    lora_weights_path=".",        # dir holding lora_weights.safetensors
)

# Plain TTS
wav = model.generate(text="สวัสดีครับ ยินดีที่ได้รู้จัก", cfg_value=2.5, inference_timesteps=10)

# Voice cloning from a 3-10 s reference clip
wav = model.generate(text="ทดสอบการโคลนเสียง", reference_wav_path="ref.wav",
                     cfg_value=2.5, inference_timesteps=10)

Or via the CLI: voxcpm clone --text "..." --reference-audio ref.wav --lora-path . -o out.wav

Limitations

• Rare archaic / liturgical Pali-Sanskrit vocabulary may occasionally be mispronounced (no phonetic-respelling dict is used — raw-text approach).
• Eval prompt sets are small; numbers are directional.
• Best for everyday/conversational Thai; not tuned for specific domains.

License

cc-by-sa-4.0 - inherited from the most restrictive training-data license (porjai_central is CC-BY-SA-4.0; Common Voice is CC0; LibriTTS-R is CC-BY-4.0). Commercial use is permitted under share-alike. The base model openbmb/VoxCPM2 is Apache-2.0; SiangTTS code is Apache-2.0.