orpheus-3b-0.1-pretrained-char-pld-ilo

orpheus-3b-0.1-pretrained-char-pld-ilo

Ilocano text-to-speech: unsloth/orpheus-3b-0.1-pretrained finetuned on the Philippine Language Dataset (PLD), read speech collected by the UP Diliman Digital Signal Processing Laboratory. Part of the halohalo project.

In plain words

This model reads Ilocano text aloud. Give it a sentence, and it produces a short audio clip of a voice from the training recordings saying it.

How good is it? When we play its output to a speech recogniser, about 92 of every 100 characters come back right (for a real human recording the figure is about 97-99). Below about 90 the speech is hard to follow.

It needs a GPU and the halohalo code to run; it is not a one-line download-and-play model yet.

This is the one to use for Ilocano text-to-speech from this organisation, as of the date on this card.

Method (for practitioners)

LoRA (r=64) on unsloth/orpheus-3b-0.1-pretrained in bf16, 2000 steps, speaker- and prompt-disjoint split. Text frontend: char. The model must be prompted the way it was trained — finetune_orpheus.frontend_text(text, 'char') in github.com/sapinsapin/halohalo — or it is reading a notation it never saw. Characters won the frontend ablation in docs/tts_sota_plan.md over BPE and syllables.

The metrics are round-trip: 50 frozen sentences synthesized, transcribed by an ASR judge, scored against the text. The human recordings of those sentences score 0.9% CER through the same judge, which is the floor; MMS-TTS scores 13.7% on the same sentences and judge. Re-transcribed by Meta's MMS-1b-all instead, as an independent judge: this model 17.5%, MMS-TTS 14.7%, human recordings 11.6%. The two judges often disagree about which of the two systems is better — ours was fine-tuned on the recordings this model learned from, Meta's comes from the project that made MMS-TTS — so that comparison is unresolved. Caveats that travel with these numbers: 50 sentences resolves a 3-point gap, not a 1-point one; and the judge is our own model trained on the same corpus, so an independent judge is still owed before anyone cites this.

Trained on PLD, which is CC-BY-NC and research-only. The base model carries Llama 3.2 terms of its own.

Evaluation

Frozen speaker- and prompt-disjoint split of PLD (splits/pld_*.json in the repo): no test speaker and no test sentence appears anywhere in training. Numbers on this split are not comparable with the in-domain figures on the dataset card, which share both, and are typically several times higher for that reason.

Round-trip intelligibility: 50 frozen test sentences are synthesized, transcribed by an ASR judge (whisper-large-v3-pld-<lang> where it exists, else whisper-small-pld-<lang>), and scored against the text. cer/wer are character and word error rates after lowercasing and stripping punctuation and accents. spk_sim is ECAPA cosine similarity to the human recording of the same sentence (1.0 = same voice). The human recordings themselves score 1-3% CER through the same judge; that is the floor.

How to use

This is a LoRA adapter on unsloth/orpheus-3b-0.1-pretrained; audio is SNAC codec tokens.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = "unsloth/orpheus-3b-0.1-pretrained"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(model, "sapinsapin/orpheus-3b-0.1-pretrained-char-pld-ilo").eval()

Then follow finetune_orpheus.py in the halohalo repo: build the prompt with frontend_text(text, "char") -- the adapter was trained on char text and must be prompted the same way -- wrap it in the SOH/EOT/EOH/SOAI/SOS framing, generate, and decode the tokens with decode_tokens through SNAC 24 kHz. scripts/tts_eval.py --stage synth does all of this.

Caveats

• 50 sentences per language resolves a gap of a few CER points, not of one.
• The judge is our own ASR model, trained on the same corpus; an independent judge is still owed before these numbers are cited.
• Read, prompted speech only. Spontaneous or noisy input is out of domain.

Licence

cc-by-nc-4.0. PLD is CC-BY-NC and research-only, so every model trained on it inherits that regardless of the base model's own licence. The base model's terms apply in addition.

Trained with finetune_tts.py from halohalo; the dataset adapter normalises each corpus to (audio@16k, text, speaker_id).