VoxCPM2-Ghana

VoxCPM2-Ghana: Multilingual Ghanaian IPA Text-to-Speech

A multilingual text-to-speech model for 42 Ghanaian languages trained on IPA phoneme inputs. Built by finetuning VoxCPM2 on Ghanaian speech with IPA transcriptions.

Try it live: HuggingFace Space

Author: Mich-Seth Owusu (@michsethowusu)

Demo Samples

The samples below cover a subset of the supported languages. The model supports IPA-based generation for all 42 Ghanaian languages plus English — use any IPA text from the supported languages with the inference script.

Listen to generated samples:

More samples (without reference audio) are in the samples/ directory.

Model Details

• Architecture: VoxCPM2 (LLaMA-like LM + AudioVAE decoder)
• Parameters: ~5B (LM) + AudioVAE
• Base model: VoxCPM2 (Microsoft)
• Finetuning: Full model finetuning (not LoRA) on 40k steps
• Sample rate: 48 kHz output (16 kHz latent encoding)
• Language support: 42 Ghanaian languages + English + code-switch
• Input format: IPA phoneme sequences (space-separated)
• Tokenizer: SentencePiece (73,440 vocab, 55,305 IPA/multichar units)

Training Details

Hyperparameters

Training Data

Source corpora:

• Bible speech recordings (JW / Waxal / UNICEF) — majority of data
• Ghana News corpus (English)
• Finance domain (Akuapem Twi)
• Code-switch datasets (Twi-English)

Training Curves

Training converged by step 40,000 with decreasing loss across all metrics.

G2P Tools & Phonemisation

All text was phonemised to IPA before training. The following G2P tools were used:

Ghanaian Languages (42 languages)

• Tool: ghana-g2p (pip install ghana-g2p), built on africa-g2p
• Convention: Space-separated IPA with punctuation kept as standalone tokens
• Coverage: 42 Ghanaian languages with language-specific grapheme-to-phoneme rules

English

• Tool: GhanaNLP/ghana-english-g2p — GhanaEnglishG2P
• Note: English uses Ghana-accented English phoneme conventions, not RP/GA. This is intentional: the model was trained on Ghanaian-accented English speech, so the IPA must match that accent.

ASR-Assisted Transcription

• Tool: Custom Ghanaian phoneme ASR (CTC, 172 phoneme vocab) finetuned to English IPA
• Purpose: Transcribe audio that lacked IPA labels, then relabel training latents
• Repo: ghanaopendata/ghana-speech-ASR-IPA

Datasets

Training Latents

• Repo: ghananlpcommunity/voxcpm2-ghana-speech-ipa-latents
• Configs: 42 language configs (balanced to 20h each)
• Columns: feat (AudioVAE latent fp16), text (IPA), text_ids (tokenized), dataset_id, split

Audio Companion

• Repo: ghananlpcommunity/ghana-speech-ipa-audio
• Purpose: Audio clips + IPA text for listening/verification

ASR Transcriptions

• Repo: ghanaopendata/ghana-speech-ASR-IPA
• Coverage: 434 shards, all 42 languages transcribed with IPA ASR

Inference

The model reads IPA, not orthography. Phonemise your text first with ghana-g2p, which uses the same rules and the same space-separated convention the model was trained on:

pip install ghana-g2p

from ghana_g2p import GhanaG2P

g2p = GhanaG2P("Asante Twi")          # any of the 42 supported languages
ipa_text = g2p.ipa("Nnipakuo bi a wɔɔyɛ adwuma", sep=" ")
# 'n n ɪ pʰ a kʰ uo b ɪ a w ɔ ɔ j ɛ a d͡ʒ u m a'

Feeding orthographic text directly will not fail loudly — it will simply produce the wrong sounds, so check that what you pass in looks like the IPA above.

from voxcpm import VoxCPM
import numpy as np, soundfile as sf

# Load model
model = VoxCPM(
    voxcpm_model_path="ghananlpcommunity/VoxCPM2-Ghana",
    optimize=False,
    device="cuda",
)

# Generate from the IPA produced above
wav = model.generate(
    text=ipa_text,
    cfg_value=2.0,
    inference_timesteps=10,
    normalize=False,  # CRITICAL: must be False for IPA input
)

sf.write("output.wav", np.array(wav).squeeze(), 48000)

With Reference Audio (Voice Cloning)

wav = model.generate(
    text=ipa_text,
    reference_wav_path="path/to/reference.wav",
    cfg_value=2.0,
    inference_timesteps=10,
    normalize=False,
)

Important Notes

• normalize=False is required. The text normaliser is designed for orthographic text and will corrupt IPA strings.
• The model outputs at 48 kHz. Writing at the wrong sample rate will pitch-shift the output.
• Voice cloning via reference_wav_path is supported (VoxCPM2 only). Use a clean 3-8 second reference clip.

Scripts

The training and data preparation scripts are included in the training/ directory:

Limitations

1. IPA input only. This model takes IPA phoneme sequences, not raw text. You need a G2P tool for each supported language to convert text to IPA before synthesis.

2. English phoneme convention. English uses a Ghana-accented IPA convention (ghana-english-g2p), not standard RP or GA. The English phoneme ASR was finetuned on g2p targets so output matches this convention.

3. Bible speech domain bias. The training data is predominantly read Bible speech (JW/Waxal). The model performs best on formal read speech and may struggle with casual/conversational prosody.

4. Background noise in English. Some English training clips had residual background noise from the source recordings. The English subset was not fully cleaned.

5. Limited training epochs. The model was trained for ~3 epochs (40k steps). Further finetuning may improve quality, especially for underrepresented languages.

6. Uneven language coverage. While data was balanced to 20h per language, some languages have more diverse speakers/domains than others. Languages like Dagbani and Ewe have richer UNICEF data; smaller languages rely primarily on Bible speech.

7. No explicit speaker modelling. The model does not condition on speaker identity. Each generation may produce a slightly different voice.

8. 48 kHz output. The model outputs 48 kHz audio. For downstream use at 16 kHz, resample after generation.

9. Single GPU inference only. The model requires ~10 GB VRAM for inference. Tested on NVIDIA H200.

10. Text normaliser disabled. normalize=False is mandatory for IPA input. Using the normaliser will corrupt phoneme sequences.

Citation

If you use this model, please cite:

@misc{ghananlp2025voxcpm2ghana,
  title={VoxCPM2-Ghana: Multilingual Ghanaian IPA Text-to-Speech},
  author={GhanaNLP},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/ghananlpcommunity/VoxCPM2-Ghana}
}

License

CC-BY-NC 4.0 (non-commercial). See LICENSE for details.

This license applies due to the non-commercial nature of the training data sources (Bible/JW speech recordings, UNICEF educational recordings). The upstream VoxCPM2 model uses Apache 2.0.