vits-ar-sa-huba

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

An advanced text-to-speech (TTS) system specifically designed for the Saudi dialect, built on the VITS architecture and utilizing the pre-trained weights from Facebook's vits ara model. The model is capable of:

Generating natural and realistic speech: Producing high-quality Saudi dialect speech that closely mimics human voices, preserving intonation and linguistic nuances. Understanding colloquial text: Processing text written in the Saudi dialect, including idiomatic expressions and local vocabulary. Controlling voice characteristics: Adjusting various aspects of the generated speech, such as pitch and speaking rate. Providing ease of use: Offering a simple and user-friendly interface for converting text to speech with high quality. Model Details VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) is an end-to-end speech synthesis model that predicts a speech waveform conditional on an input text sequence. It is a conditional variational autoencoder (VAE) comprised of a posterior encoder, decoder, and conditional prior.

A set of spectrogram-based acoustic features are predicted by the flow-based module, which is formed of a Transformer-based text encoder and multiple coupling layers. The spectrogram is decoded using a stack of transposed convolutional layers, much in the same style as the HiFi-GAN vocoder. Motivated by the one-to-many nature of the TTS problem, where the same text input can be spoken in multiple ways, the model also includes a stochastic duration predictor, which allows the model to synthesise speech with different rhythms from the same input text.

Usage

MMS-TTS is available in the 🤗 Transformers library from version 4.33 onwards. To use this checkpoint, first install the latest version of the library:

pip install  transformers[torch]

Then, run inference with the following code-snippet:

from transformers import VitsModel, AutoTokenizer
import torch

model = VitsModel.from_pretrained("wasmdashai/vits-ar-sa-huba")
tokenizer = AutoTokenizer.from_pretrained("wasmdashai/vits-ar-sa-huba")

text = "السلام عليكم  كيفك  عساك بخير  "
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
  full_generation =model(**inputs)
full_generation_waveform = full_generation.waveform.cpu().numpy().reshape(-1)

from IPython.display import Audio

Audio(full_generation_waveform, rate=model.config.sampling_rate)

Output full_generation_waveform

Contact

You can also email us at [email protected]

مجموعة نماذج توليد اللهجات العربية

مقدمة

يسرنا أن نعلن عن إصدار مجموعة من نماذج توليد اللهجات العربية قريبًا. تم تصميم هذه النماذج باستخدام تقنيات الذكاء الاصطناعي المتقدمة لتقديم تجربة طبيعية وواقعية في تحويل النص إلى كلام (Text-to-Speech) بمختلف اللهجات العربية.

جدول النماذج

التفاصيل الفنية

تعتمد جميع النماذج على بنية VITS، وهي نموذج شامل لتحويل النص إلى كلام يتيح توليد موجات صوتية واقعية بناءً على المدخلات النصية. تحتوي النماذج على محولات لتحليل النص وتوليد الكلام بناءً على خصائص الصوت المحلية لكل لهجة.

الترقيات المستقبلية

سيتم تقديم تحديثات منتظمة لتحسين جودة الصوت وزيادة كفاءة فهم اللهجات المختلفة. تابعونا لمعرفة المزيد حول تواريخ الإطلاق الدقيقة لكل نموذج.

Acknowledgements

This implementation is based on tts-arabic, VITS, Finetune VITS and Bert-VITS2. We appreciate their awesome work.