mms-tts-asante-twi-bibletts

mms-tts-asante-twi-bibletts

Prototype. Non-commercial. Do not build production work on this.

A VITS text-to-speech model for Asante Twi, fine-tuned from facebook/mms-tts-aka on a single studio narrator, with the tokenizer vocabulary extended so the model can learn phrase breaks from punctuation.

Part of LIBOG, a free speech toolkit for Ghanaian languages.

Read this before you use it

The tokenizer is loaded with normalize=False, and the model was trained on text cleaned a specific way. Feeding it raw text will not match what it learned and the output will degrade silently — nothing raises. The cleaning is: Unicode NFC, map smart punctuation to ASCII, lowercase, replace every character outside the vocabulary with a space, then collapse whitespace. Punctuation the vocabulary does contain is kept deliberately, because that is what carries the phrasing.

This snippet is complete and runnable as written:

import unicodedata
import torch
from transformers import VitsModel, VitsTokenizer

repo = "danieldzikunuofmarvel/mms-tts-asante-twi-bibletts"
model = VitsModel.from_pretrained(repo)
tok = VitsTokenizer.from_pretrained(repo, normalize=False)

# Read the vocabulary off the encoder, NOT get_vocab(): get_vocab enumerates
# range(len(encoder)), which stops one short here and omits ";".
VOCAB = set(tok.encoder)

REPLACE = {"‘": "'", "’": "'", "“": "", "”": "",
           "–": "-", "—": "-", " ": " ", "…": "."}


def clean_for_tts(text):
    text = unicodedata.normalize("NFC", text)
    for a, b in REPLACE.items():
        text = text.replace(a, b)
    text = text.lower()
    kept = [ch if ch in VOCAB else " " for ch in text]
    return " ".join("".join(kept).split())


text = clean_for_tts("Awurade ne me hwɛfoɔ, biribiara renhia me.")
inputs = tok(text, return_tensors="pt")
with torch.no_grad():
    audio = model(**inputs).waveform  # 16 kHz

Output is 16 kHz mono, matching the MMS base.

Intended use

Free, non-commercial use: language learning, accessibility tooling, research on low-resource Ghanaian speech, and demos.

Out of scope: anything commercial (see Licence), and any setting where a mispronunciation carries real cost. It is trained on scripture read by one narrator, so it speaks in that register and that voice. It is not a general-purpose Twi voice and it does not represent Akuapem Twi, Fante, or any other Akan variety.

Training

The vocabulary was extended from 30 to 37 tokens to add punctuation, and the new embedding rows were hand-seeded to match the trained embeddings' standard deviation rather than left at the much narrower default initialisation.

The dataset is a repair, not the original

BibleTTS Asante Twi as published had the characters ɛ and ɔ zeroed out across the corpus. The training data here is a repaired version — 118,154 ɛ and 103,272 ɔ restored — published separately and reported upstream.

Evaluation

There is no automatic metric here, by design. Success was defined as a Twi speaker listening to held-out sentences and judging them correct.

That test was run and passed: the voice was judged clearly better than the unmodified facebook/mms-tts-aka base, though not flawless. This is an informal judgement by a single listener, not a measured MOS study, and should be read as such.

One caveat worth stating plainly: the duration loss — the component that tracks phrasing — stayed flat throughout training. With a stochastic duration predictor that is closer to normal than to broken, but it means the training log does not demonstrate that the added punctuation tokens learned anything. Whether punctuation genuinely improves phrase breaks in this model is not yet established by measurement.

Limitations

• Single narrator, single register (scripture). Expect degradation on conversational text.
• Asante Twi only.
• Punctuation phrasing is unverified by measurement (above).
• Numerals are not handled. The MMS Akan vocabulary contains 2 and 3 as tone markers, so those two survive text cleaning while other digits are dropped: 12345 becomes 23. Expand numbers to Twi words before synthesising.
• Not evaluated for pronunciation of names, loanwords, or code-switched English.

Licence

CC BY-NC 4.0, inherited from facebook/mms-tts-aka. Non-commercial use only. Meta released MMS under plain CC BY-NC with no commercial licensing path, so commercial use would mean retraining on a permissive base, not buying a licence.

An unresolved question, flagged rather than answered: BibleTTS is CC BY-SA 4.0, which requires derivatives be shared alike, and CC BY-SA and CC BY-NC are not compatible. Whether trained model weights constitute a derivative work of their training data is genuinely unsettled law. This model is released non-commercially on the conservative reading. If that matters for your use, get advice from someone qualified to give it.

Credits

• facebook/mms-tts-aka — Meta AI, Massively Multilingual Speech
• BibleTTS — Masakhane / Biblica, CC BY-SA 4.0
• Fine-tuning recipe — ylacombe/finetune-hf-vits