chatterbox-afaan-oromo

Chatterbox Afaan Oromo

A LoRA adapter that teaches Chatterbox Multilingual v3 (Resemble AI, MIT) to speak Afaan Oromo, with voice cloning from about ten seconds of reference audio. Trained only on speech that is licensed for it.

Stock Chatterbox can read Qubee: it's Latin script, and the tokenizer covers every letter. What it doesn't have is the language. Fed an Afaan Oromo sentence it produces something with the right letters and the wrong sounds: dh, ph, ny, x, q, the long vowels and the glottal-stop apostrophe (ta'uu) all come out as whatever its other languages taught it. The adapter teaches it what those spellings sound like when an Afaan Oromo speaker says them. It also often falls apart near the end of a clip; that goes away too. Qubee needs no new tokens: the base tokenizer already covers a–z and the apostrophe.

The repo also has afaan_oromo_text.py, the text normalizer the model was trained through. No dependencies, works on its own (below).

Hear it

Each pair uses the same sentence, reference audio, settings and seed. No language tag on either: the adapter was trained without one and the base has no [om].

Reading this on GitHub? The players only render on Hugging Face. Click a clip in demo/ to play it, or watch demo/before_after.mp4 (all eight pairs). The weights (new_lang_adapter/) are only on Hugging Face; everything else is mirrored here.

Texts: demo/sentences.txt. Reference voice: demo/reference.wav, a held-out speaker from the WaxalNLP Afaan Oromo set (CC-BY-SA-4.0; never in training) (<audio controls preload="none" src="https://huggingface.co/gabar-tech/chatterbox-afaan-oromo/resolve/main/demo/reference.wav"></audio>). Both models got the text after afaan_oromo_text.normalize, temperature=0.6, cfg_weight=0.5, seed 1234. One take per sentence per model, no picking. Known weaknesses are under Limitations.

What this is, and what it isn't

An adapter: LoRA weights on the T3 text-to-speech-token transformer plus the (re-trained) text embedding tables. You apply it on top of Chatterbox Multilingual v3, which you download from Resemble. We ship our delta, not a copy of their model.

Not merged weights, not a standalone model, not a production service. Afaan Oromo only.

Afaan Oromo text front-end

afaan_oromo_text.py built the training labels, and the loader runs it on every input, so training and inference see the same text. One file, standard library only, same licence as the adapter, usable without the model.

It spells out numbers, ordinals, percentages and currency in Afaan Oromo (5876 → kuma shaniif dhibba saddeetiif toorbaatamii ja'a, 3ffaa → sadaffaa, %20 → dhibbeentaa digdama, $100 → doolaara dhibba tokko), lower-cases (the base model was trained on lower-cased text; Qubee capitalisation isn't phonemic), keeps the apostrophe (a letter in Qubee: the glottal stop in hundaa'u, wal'aansi), reduces punctuation to . , ; ? ! and strips URLs, emoji and control characters. Years read as cardinals.

from afaan_oromo_text import normalize, split_sentences
normalize("Bara 2018 filannoo irratti Caaltuun sagalee %75 argatte.")
# "bara kuma lamaaf kudhan saddeet filannoo irratti caaltuun sagalee dhibbeentaa toorbaatamii shan argatte."

Training data

Two sources. Every clip's filename starts with its corpus prefix; the assembled training directory was audited before training and the output is committed as is (audit/corpus_audit.txt). The adapter was trained from scratch on exactly that directory, starting from Resemble's stock v3 T3.

WaxalNLP comes as 48 kHz and was resampled down to 24 kHz; the Mendeley set is 24 kHz already. All Waxal Afaan Oromo clips that fit the trainer's 3 to 25 second window were used; the Mendeley corpus adds one studio voice.

Licence: why CC-BY-SA-4.0

WaxalNLP is licensed under CC BY-SA 4.0, so the adapter is released under CC BY-SA 4.0. Credit to Digital Umuganda, the WaxalNLP contributors, and the authors of the Mendeley corpus. The base model is MIT; this licence covers what we add.

Architecture

Base: Chatterbox Multilingual v3, ResembleAI/chatterbox at revision 5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18, T3 file t3_mtl23ls_v3.safetensors. Pinned on purpose: the adapter only makes sense on that exact T3. v3 has Resemble's hallucination and speaker-similarity fixes over v2; S3Gen, the voice encoder and the tokenizer are the same as v2 and untouched.

Notes for anyone building on this:

About those 14 tokens. They come from nine bad transcripts, not from the language. Two clips — wxlom_006638 and wxlom_008188 — have a run of Amharic letters typed after the sentence's final full stop, the way a keyboard left in the wrong layout will do it: … hin turu. followed by አየከየኸየ. Nothing in the audio corresponds to them. Seven more clips have one wrong letter inside a word — ķan for kan, bakkeeŵwan for bakkeewwan, one stray superscript zero. Between them those nine transcripts introduced ten characters the base tokenizer had never seen, and four Ge'ez punctuation marks followed because our tokenizer adds them as soon as it sees any Ethiopic character at all.

We don't think this affects the model. Nine labels out of 49,947 is nothing, the new rows barely move during training, and the model has no reason to produce a character it has never been asked for — so we left them rather than spend another twelve hours of GPU time on tidiness. Anyone retraining on this corpus can clean it in a minute: drop those nine .lab files, or strip whatever follows the last Qubee character.

No language token. The base tokenizer has [fr], [de], [sw] and so on; there is no [om] and we didn't add one. The adapter was trained on plain normalized (lower-cased) text, and the loader tokenizes without a language prefix. language_id="om" on stock ChatterboxMultilingualTTS.generate raises ValueError; use the loader.

Alignment guard. Upstream enables its attention-alignment hallucination guard only when text_tokens_dict_size == 2454. With the extended vocabulary it's off, in training and at inference. The loader chunks by sentence, which handles the common failure (T3 stopping at the first sentence-final mark).

Train/inference parity. The training labels and the loader use the same afaan_oromo_text.py. Both hashes are recorded in training_config.json as label_frontend_sha256 and label_frontend_sha256_shipped. Property checks over every training transcript are in audit/frontend_check.txt.

Evaluation

Held-out set: 100 clips from the same corpus, split before training by a seeded speaker-disjoint rule, so whole speakers are held out and none of their sentences appear in training. Checked independently of the trainer's own assertion: HELD OUT: eval ∩ train = ∅ at clip, speaker and sentence level; all eval stems are oro_/wxlom_. (audit/holdout_verify.txt). Both models ran on the same clips with the same per-clip reference audio (the held-out speaker's own recording), through the same code: the released loader for the adapter, stock v3 with the same normalized text and no language tag.

Per-clip numbers: audit/eval/.

How CER is measured. We transcribe the generated audio with Meta's stock omniASR-CTC-3B (orm_Latn), which we didn't train, and compare to the reference text after case-folding and stripping punctuation; the word-internal apostrophe is kept, because dropping a glottal stop is a real pronunciation error. Same ASR, same normalization, both models. UTMOS and ECAPA involve no ASR. UTMOS was trained on English MOS ratings and both outputs are Afaan Oromo, so treat it as a rough signal. Means are over clips that produced audio; failures are on their own row so they can't hide in an average.

Listening verdict: a native Afaan Oromo speaker called the difference large and unambiguous, in the adapter's favour. The stock model reads Qubee with an audibly foreign accent and frequently breaks down near the end of a clip; we heard none of that from the adapter, which sounds like Afaan Oromo spoken rather than spelled out.

Usage

pip install "chatterbox-tts @ git+https://github.com/resemble-ai/chatterbox@5de7a54aa4e5e2baadb0182dde554908b48b85c2" peft safetensors huggingface_hub torchaudio

from huggingface_hub import hf_hub_download
import importlib.util, torchaudio

# the loader + text front-end ship in this repo
spec = importlib.util.spec_from_file_location(
    "afaan_oromo_tts", hf_hub_download("gabar-tech/chatterbox-afaan-oromo", "afaan_oromo_tts.py"))
afaan_oromo_tts = importlib.util.module_from_spec(spec); spec.loader.exec_module(afaan_oromo_tts)

tts = afaan_oromo_tts.load_afaan_oromo_tts(device="cuda")   # downloads base v3 (pinned) + adapter
wav = tts.generate(
    "Akkam jirta? Kun sagalee Afaan Oromoo barreeffama irraa kallattiin uumame dha. Har'a Hagayya 11 bara 2018 dha.",
    audio_prompt_path="reference.wav",     # ~10 s of the voice to clone, with consent
    temperature=0.6, cfg_weight=0.5)
torchaudio.save("out.wav", wav, tts.sr)      # 24 kHz, PerTh-watermarked
print(tts.normalize("Har'a Hagayya 11 bara 2018 dha."))   # what the model actually read

Or from a checkout: python afaan_oromo_tts.py "Nagaa hundaaf." --ref reference.wav --out out.wav.

generate() normalizes the text, splits at sentence-final marks (T3 tends to stop at the first . / ? / !), synthesizes each sentence against the reference and joins them. normalize=False / split_sentences=False turn those off. We ran the snippet above as written in a fresh virtualenv with only the packages listed, on NVIDIA RTX A6000, Ubuntu 22.04.5 LTS, python 3.11.10, torch 2.6.0+cu124, chatterbox-tts@5de7a54, peft 0.20.0, before publishing.

Watermarking

Chatterbox puts Resemble's PerTh watermark in every waveform it generates. We left that alone; nothing in this adapter touches S3Gen or the vocoder, where it happens. We ran the public resemble-perth detector over every demo clip from both models and the held-out eval outputs, with the natural reference recording as a negative control: present on all 116 checked files (detector confidence ≥ 0.5 on every generated file; the natural reference recording scores 0.0, so the detector is discriminating) (audit/watermark_verify.txt). If you build on this, leave it in.

Intended use

• Afaan Oromo speech interfaces, audiobooks, education, accessibility, media production, with the consent of whoever's voice you clone.
• Research on low-resource TTS and Ethiopian language tech.

Out of scope

• Cloning someone's voice without their informed consent.
• Political persuasion, impersonating public figures, fraud (voice authentication included), harassment.
• Other languages. The adapter makes the base worse at its other languages.
• Anything safety-critical or broadcast without a human listening first.

Limitations

• Numbers, ordinals, percentages and currency are expanded by afaan_oromo_text.py. Skip it, or feed it things it doesn't handle (dates in odd formats, clock times, English words), and the model gets them raw.
• The training transcripts are crowdsourced and carry occasional typos. We found nine and left them (see Architecture); we did not go looking for the rest, so assume there are more.
• Consonant length and vowel length are written in Qubee (baddaa / bada), and the model learns them from spelling; it will still get some wrong.
• The first word or two of an utterance are sometimes slurred before the model settles; mid-sentence text is steadier.
• Very short inputs (a single word) can come out unstable. Put them in a sentence.
• Long passages: chunk at sentence boundaries. The loader does this for you.
• 209.92 hours, almost all of it read speech (Waxal is an ASR corpus; the Mendeley set is read studio speech). Expect a neutral, read-aloud delivery; expressive and spontaneous speech is thin.
• Afaan Oromo only.

Risks and misuse

This model can clone a voice from roughly ten seconds of reference audio, so it can be misused for impersonation or fraud. Use it only with informed consent and human review. Outputs are watermarked by the shipped inference path. Every voice in the training data was recorded under a licence that permits this use.

Attribution

• Resemble AI, for Chatterbox Multilingual (MIT), the base model and the PerTh watermarker.
• Digital Umuganda and the WaxalNLP contributors, Afaan Oromo speech data (CC-BY-SA-4.0).
• The authors of the Afaan Oromo Text-to-Speech Synthesis dataset (Mendeley Data, mpy85ns82z, CC-BY-4.0).

Citation

@misc{gabar2026chatterboxoromo,
  title  = {Chatterbox Afaan Oromo: an Afaan Oromo adapter for Chatterbox Multilingual},
  author = {{Gabar Technologies}},
  year   = {2026},
  url    = {https://huggingface.co/gabar-tech/chatterbox-afaan-oromo}
}