omnivoice-lora-zh

OmniVoice LoRA — Chinese (zh)

Fine-tuned LoRA adapter for OmniVoice to improve zero-shot voice cloning quality for Chinese.

Training Details

• Base model: k2-fsa/OmniVoice (Qwen3-0.6B backbone)
• Method: LoRA (rank=32, alpha=64, RSLoRA)
• Target modules: Self-attention + audio projection layers
• Training data: Best-of-N distilled Chinese speech samples
• Steps: 200
• Precision: bf16
• Hardware: NVIDIA A40 (48GB)

Usage

from omnivoice import OmniVoice
from peft import PeftModel
import torch

# Load base model
model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16,
)

# Load LoRA adapter
model.llm = PeftModel.from_pretrained(model.llm, "amanuelbyte/omnivoice-lora-zh")
model.llm = model.llm.merge_and_unload()

# Generate
audio = model.generate(
    text="Your chinese text here",
    ref_audio="path/to/reference.wav",
)

IWSLT 2026

This adapter was developed for the IWSLT 2026 shared task on cross-lingual voice cloning.