matxa-tts-v2-ca-multiaccent-graphemes

🍵 Matxa-TTS (Matcha-TTS) v2 Multi-Accent Catalan Graphemes

Table of Contents

<details> <summary>Click to expand</summary>

• Summary
• Model description
• Intended uses and limitations
• How to Use
• Training
• Evaluation
• Citation
• Additional information

</details>

Summary

Here we present Matxa-TTS 🍵 v2: a Catalan multispeaker and multi-accent neural TTS model working with graphemes. It works together with a vocoder model to generate high quality and expressive speech efficiently. You can use any vocoder such as 🥑 alVoCat or Wavenext.

Model Description

Matxa-TTS 🍵 is based on Matcha-TTS, an encoder-decoder architecture designed for fast acoustic modelling in TTS. The encoder part is based on a text encoder and a phoneme duration prediction that together predict averaged acoustic features. The decoder has essentially a U-Net backbone inspired by Grad-TTS, which is based on the Transformer architecture. By replacing 2D CNNs by 1D CNNs in the decoder, a large reduction in memory consumption and fast synthesis is achieved.

Matxa-TTS is a non-autorregressive model trained with optimal-transport conditional flow matching (OT-CFM). This yields an ODE-based decoder capable of generating high output quality in fewer synthesis steps than models trained using score matching.

Modifications

1. Decoder's number of channels and attention head dimension were increased to 512 and 128, respectively
2. Two timestep sampling techniques for diffusion were added: (1) stratified sampling, where each batch sample is assigned a different segment of [0,1) to guarantee uniform coverage of diffusion time; and (2) a cosine timestep scheduler
3. A denoiser is implemented on the generated spectrogram during inference

For more details, please, check our new github repository branch.

Intended Uses and Limitations

This model is intended to serve as an acoustic feature generator for multispeaker text-to-speech systems for the Catalan language. It has been finetuned using a Catalan text, therefore if the model is used for other languages it will not produce intelligible samples after mapping its output into a speech waveform.

The quality of the samples can vary depending on the speaker. This may be due to the sensitivity of the model in learning specific frequencies and also due to the quality of samples for each speaker.

For more information see the licenses section under Additional information.

How to Use

Installation

Create a virtual environment:

python -m venv /path/to/venv
source /path/to/venv/bin/activate

Clone the repository:

git clone -b v2-graphemes --single-branch https://github.com/langtech-bsc/Matcha-TTS.git
cd Matcha-TTS

Install the package from source:

pip install -e .

Test on Collab

Try out Matxa-TTS v2 with graphemes with AlVoCat!

Note that input texts, if containing any numbers, need to be normalized!

<a target="_blank" href="https://colab.research.google.com/drive/12Wki0uuWWipAcQA-lxtnvAGKey9SVrXi?usp=sharing"> <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> </a>

Know more about our Catalan voices (and their corresponding IDs in parenthesis):

• LaFresCat

• EnVeuAlta

• ReFresCat

Note: The names shown are not the real speakers’ names.

End-to-End ONNX Version

We provide an end-to-end Catalan multi-accent Matxa-TTS v2 in ONNX format. This Catalan multi-accent Matxa-TTS v2 was exported to ONNX, and then concatenated with our 🥑 alVoCat (WaveNext ONNX version).

Training

This model has been trained from scratch with LaFresCat dataset together with other studio-quality Catalan multi-accent recordings (ReFresCat and EnVeuAlta).

Using this set of hyperparameters:

• Learning Rate → 5e-5
• Scheduler → Cosine scheduler with a minimum learning rate 3e-5
• Batch Size → 32
• GPUs → 2
• Num. Steps → 130K

Evaluation

We evaluated the model in terms of quality and intelligibility using a test set of 350 sentences extracted from La creació d'Eva i altres contes by Josep Carner i Puig-Oriol, available on Project Gutenberg. Speech quality was assessed using ScoreQ, a recent approach for evaluating synthesized speech. Intelligibility was measured by computing Word Error Rate (WER) and Character Error Rate (CER) using the whisper-bsc-large-v3-cat model, which was trained and evaluated on multi-accented Catalan data.

We used AlVoCat as a vocoder, already trained on Catalan data. For comparison, we evaluated the model together with the same model trained with phonemes using the same set of sentences. When training directly with graphemes, the model achieves better scores in terms of intelligibility while being competitive with the phoneme version in terms of audio quality. In addition, Matxa-TTS-v2-ca-multiaccent-graphemes does not require a phonemizer, making the model more flexible and easier to integrate.

Citation

If this code contributes to your research, please cite the work:

@misc{LangtechVeu2025matxattsv2multispeaker,
      title={Grapheme-Based Catalan Multi-Accent Text-to-Speech}, 
      author={Alex Peiró-Lilja and Jose Giraldo},
      organization={Barcelona Supercomputing Center},
      url={huggingface.co/BSC-LT/matxa-tts-v2-ca-multiaccent-graphemes},
      year={2026}
}

Additional Information

Author

The Speech Team at the AI Institute, Barcelona Supercomputing Center.

Contact

For further information, please send an email to [email protected].

Copyright

Copyright(c) 2025 by the AI Institute, Barcelona Supercomputing Center.

License

This model is free to use for non-commercial and research purposes (see LICENSE.md for full terms). Commercial use is only possible through licensing by the voice artists. For further information, contact [email protected] and [email protected].

Funding

This work is funded by the Ministerio para la Transformación Digital y de la Función Pública - Funded by EU – NextGenerationEU within the framework of the project ILENIA with reference 2022/TL22/00215337.

The conversion of the model was possible thanks to the computing time provided by Barcelona Supercomputing Center through MareNostrum 5.