soloba-tdt-0.6b-v1.5

Soloba-TDT-600M Series

<style> img { display: inline; } </style>

| |

soloba-tdt-0.6b-v1.5 is a fine tuned version of RobotsMali/soloba-tdt-0.6b-v0.5 on RobotsMali/kunkado. This model does not consistently produce Capitalizations and Punctuations and it cannot produce acoustic event tags like those found in Kunkado its transcriptions. It was fine-tuned using NVIDIA NeMo.

🚨 Important Note

This model, along with its associated resources, is part of an ongoing research effort, improvements and refinements are expected in future versions. Users should be aware that:

• The model may not generalize very well across all speaking conditions and dialects.
• Community feedback is welcome, and contributions are encouraged to refine the model further.

NVIDIA NeMo: Training

To fine-tune or play with the model you will need to install NVIDIA NeMo. We recommend you install it after you've installed latest PyTorch version.

pip install nemo-toolkit['asr']

How to Use This Model

Note that this model has been released for research purposes primarily.

Load Model with NeMo

import nemo.collections.asr as nemo_asr
asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="RobotsMali/soloba-tdt-0.6b-v1.5")

NeMo 2.5.0 Compatibility

This checkpoint was created with NeMo 2.5.0. Loading it with newer NeMo versions (observed with 2.7.x) can fail because the strict decoding schema expects key_phrase_items_list; see NVIDIA-NeMo/Speech#15658. This workaround was tested with Python 3.12:

from pathlib import Path

from nemo.collections.asr.models import ASRModel
from omegaconf import OmegaConf

model_name = "RobotsMali/soloba-tdt-0.6b-v1.5"
cfg = ASRModel.from_pretrained(model_name, return_config=True)
OmegaConf.set_struct(cfg, False)

for decoder in ("greedy", "beam"):
    boosting_tree = OmegaConf.select(cfg, f"decoding.{decoder}.boosting_tree")
    if boosting_tree is not None:
        boosting_tree.key_phrase_items_list = None

config_path = Path("patched_config.yaml").resolve()
OmegaConf.save(cfg, config_path)
asr_model = ASRModel.from_pretrained(
    model_name=model_name,
    override_config_path=str(config_path),
    strict=False,
)
config_path.unlink()

Transcribe Audio

asr_model.eval()
# Assuming you have a test audio file named sample_audio.wav
asr_model.transcribe(['sample_audio.wav'])

Input

This model accepts any mono-channel audio (wav files) as input and resamples them to 16 kHz sample rate before performing the forward pass

Output

This model provides transcribed speech as an hypothesis object with a text attribute containing the transcription string for a given speech sample. (nemo>=2.3)

Model Architecture

This model uses a FastConformer encoder and an autoregressive Token-and-Duration Transducer decoder, a variant of RNN-T that predicts jointly learn to predict a token and its duration. FastConformer is an optimized version of the Conformer model with 8x depthwise-separable convolutional downsampling. You may find more information on the details of FastConformer here: Fast-Conformer Model.

Training

The NeMo toolkit was used for finetuning this model for 40,000 steps over RobotsMali/soloba-tdt-0.6b-v0.5 model with batch size 32. The finetuning codes and configurations can be found at RobotsMali-AI/bambara-asr.

The tokenizer for this model was trained on the text transcripts of the train set of RobotsMali/kunkado using this script.

Dataset

This model was fine-tuned on the kunkado dataset, the human-reviewed subset, which consists of ~40 hours of transcribed Bambara speech data. The text was normalized with the bambara-normalizer prior to training, normalizing numbers, removing punctuations and removing tags.

Performance

We report the Word Error Rate (WER) and Character Error Rate (CER) for this model:

License

This model is released under the CC-BY-4.0 license. By using this model, you agree to the terms of the license.

---

Feel free to open a discussion on Hugging Face or file an issue on GitHub for help or contributions.