all-models

lina-speech (beta)

Exploring "linear attention" for text-to-speech.

It predicts audio codec "à la" MusicGen : delayed residual vector quantizers so that we do not need multiple models.

Featuring RWKV, Mamba, Gated Linear Attention.

Compared to other LM TTS model :

• Can be easily pretrained and finetuned on midrange GPUs.
• Tiny memory footprint.
• Trained on long context (up to 2000 tokens : ~27s).

Models

Installation

Following the linear complexity LM you choose, follow respective instructions first:

• For Mamba check the official repo.
• For GLA/RWKV inference check flash-linear-attention.
• For RWKV training check RWKV-LM

Acknowledgment

• The RWKV authors and the community around for carrying high-level truly opensource research.
• @SmerkyG for making my life easy at testing cutting edge language model.
• @lucidrains for its huge codebase.
• @sustcsonglin who made GLA and FLA.
• @harrisonvanderbyl fixing RWKV inference.

Cite

@software{lemerle2024linaspeech,
  title  = {LinaSpeech: Exploring "linear attention" for text-to-speech.},
  author = {Lemerle, Théodor},
  url    = {https://github.com/theodorblackbird/lina-speech},
  month  = april,
  year   = {2024}
}

IRCAM

This work takes place at IRCAM, and is part of the following project : ANR Exovoices

<img align="left" width="200" height="200" src="logo_ircam.jpeg">