orpheus-3b-lj-voice-data-efficiency

Orpheus-3B voice-clone LoRA — data-efficiency sweep (LJSpeech target)

8 LoRA adapters fine-tuning canopylabs/orpheus-3b-0.1-ft to clone the LJSpeech voice, each trained on a different amount of target audio (1, 2, 5, 10, 15, 30, 45, 60 minutes). Produced in ~1 hour on a single io.net 8xH200 node — 8 independent LoRA jobs, one per GPU (embarrassingly parallel).

Headline finding

Voice identity saturates fast — ~10 minutes of audio closes ~85% of the gap to real speech, and more barely moves it. But intelligibility keeps improving — word-error-rate falls from 58% (1 min, "voice-matched mumbling") to ~3% (45-60 min). Rule of thumb: ~10 min to clone the voice, ~30-60 min for clean speech in it.

Metric = speaker-embedding cosine (WavLM-base-plus-sv) vs a held-out real-speaker enrollment; WER = Whisper-small. 30 clips held out and never trained.

Files

• adapters/{N}min/ — LoRA adapters (r=32, attention+MLP). Attach to the base model.
• samples/ — the same held-out sentence synthesized at base / 1 / 10 / 60 min, plus the real target voice.
• voice_curve.json — full metrics per budget.

Usage

from transformers import AutoModelForCausalLM
from peft import PeftModel
m = AutoModelForCausalLM.from_pretrained("canopylabs/orpheus-3b-0.1-ft", torch_dtype="bfloat16").cuda()
m = PeftModel.from_pretrained(m, "Gurunath/orpheus-3b-lj-voice-data-efficiency", subfolder="adapters/60min")
# generate audio tokens from [SOH]+text+[EOH,SOA,SOS], stop at EOS(128258), decode with hubertsiuzdak/snac_24khz

LJSpeech is public domain. Base model Apache-2.0.