bam-vits-fintech

Bambara VITS — FinTech adaptation

RobotsMali/bam-vits-fintech continues training RobotsMali/bam-vits on RobotsMali/finBamSpeech, a higher-quality single-speaker corpus of 800 read Bambara sentences about finance, banking, and financial technology.

Research checkpoint — substantially undertrained. The already undertrained base received only 150 additional epochs (750 training examples, batch size 80). This is still far below the hundreds of thousands of optimizer steps commonly used for VITS; approximately 200,000 steps is our reference budget. No objective or human evaluation is available.

Model details

• Architecture: VITS with a HiFi-GAN-style waveform decoder
• Checkpoint: inference-only transformers.VitsModel; discriminator removed
• Input: lowercased plain Bambara orthography
• Audio: 22,050 Hz
• Training sequence: English VCTK checkpoint → afvoices-notag base → finBamSpeech domain adaptation
• Code: RobotsMali-AI/vits-bam, adapted from ylacombe/finetune-hf-vits

Usage

import soundfile as sf
import torch
from transformers import AutoTokenizer, VitsModel

repo_id = "RobotsMali/bam-vits-fintech"
text = "Juru sarali waati sera."
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = VitsModel.from_pretrained(repo_id)
inputs = tokenizer(text.lower(), return_tensors="pt")

with torch.no_grad():
    # Single-speaker adaptation omitted speaker conditioning during training.
    waveform = model(**inputs).waveform[0]

sf.write("fintech-bambara.wav", waveform.cpu().numpy(), model.config.sampling_rate)

Why num_speakers is still 20

The configuration retains the base model's 20-speaker table because the training script deliberately does not resize or reinitialize speaker embeddings when it detects exactly one target speaker. More importantly, preprocessing adds speaker_id only when more than one speaker is present. FinBamSpeech batches therefore used speaker_id=None, and VITS bypassed the embedding table during this adaptation.

The checkpoint was thus adapted as an effectively unconditioned single-speaker model while retaining multi-speaker architecture metadata. Its 20 embedding rows do not represent 20 FinTech-adapted voices. Direct inspection of the published checkpoint shows that the rows are not bit-identical, although their variance is small and they provide very similar conditioning. For inference that matches the training path, omit speaker_id as shown above.

Training data and procedure

The base checkpoint used 21,253 train and 1,128 test examples from the top 20 speakers in afvoices-notag, a noisy spontaneous-speech selection intended only for experimentation. This variant was then trained for 150 more epochs with per-device batch size 80 on FinBamSpeech's 750-train/50-test split. Its read, domain-specific recordings are higher quality than the base data, but 800 samples remain very small for VITS.

Text is lowercased plain Bambara; audio is processed at 22.05 kHz. The base hyperparameters and training code are documented in config/bam-vits.yaml.

Evaluation and comparison

No MOS, intelligibility, domain-term accuracy, speaker-similarity, or automated TTS metrics are reported. Domain adaptation should not be interpreted as proven improvement. In informal comparison, plain-orthography checkpoints often sounded slightly more natural than pseudo-IPA variants, with no remarkable pseudo-IPA improvement in convergence or quality.

Intended use, risks, and limitations

This model is for research on low-resource and domain-adapted Bambara TTS. It may be useful for qualitative experiments with finance-related text, but it is not validated for financial advice, customer communication, accessibility, or production banking systems. Never treat synthesized speech as accurate financial information.

Short training and narrow data can cause noise, poor prosody, incorrect numbers or financial terms, speaker leakage, memorization, and failures on conversational, code-switched, long, or out-of-domain text. No systematic safety, bias, privacy, or memorization testing was performed. Do not impersonate speakers or create deceptive audio; disclose that output is synthetic.

Related resources

• bam-vits: plain-orthography base
• bam-vits-pseudo-ipa-fintech: pseudo-IPA counterpart
• VITS paper

Questions are welcome in the project repository or this model's Community tab.