jaspers-rave-models

Jasper's RAVE Models

A collection of RAVE models trained by me, using RAVE (Realtime Audio Variational autoEncoder) from acids-ircam, article: https://arxiv.org/abs/2111.05011. Used in some of my own projects, including nn.terrain latent terrain synthesis, soundwalking the latent space workshop, the sketch-to-sound demo.

Supported MaxMSP externals:

• For nn~, models are the *.ts files in the root folder.
• For neural.live~, models are the *.pte programs and *.json metadata in the neural_tilde_coreml folder.

All models work with the interactive prior nn.terrain~.

All models have forward, encode, and decode methods.

Important Note

When exporting, the Gaussian noise in the encoder's reparametrize layer is removed.

Model Descriptions

Please see description for each model for training datasets, training configurations, and exporting configurations.

guitar_picking_dm_b2048_r44100_z8_causal
Training data
Acoustic and electric guitar fingerpicking in the D minor scale. Collected from released/unreleased dance and electronic tracks I made under the name Alaska Winter. Mostly self-recorded, with some additional samples from Splice sound library. Unmixed, recorded dry, approximately 2h in total after trimming silence.
Configuration
RAVE v2, 44.1kHz, block size 2048, causal, variational regularization. Exported with 8 latent dimensions, streaming is enabled.

gtsinger_b2048_r44100_z16_noncausal
Training data
A subset of the singing voice in the GTSinger dataset. Recordings of spoken lyrics were removed, all other recordings (with/without singing techniques, control groups) were included. Approximately 26h after trimming silence.
Configuration
RAVE v2, 44.1kHz, block size 2048, noncausal, Wasserstein regularization. Exported with 16 latent dimensions, streaming is enabled.

aam_drum_b2048_r44100_z16_noncausal
Training data
All drums recordings in the Artificial Audio Multitracks dataset. Approximately 104h in total after trimming silence.
Configuration
RAVE v2, 44.1kHz, block size 2048, noncausal, Wasserstein regularization, capacity 128. Exported with 16 latent dimensions, streaming is enabled.

aam_bass_b2048_r44100_z16_noncausal
Training data
All bass recordings in the Artificial Audio Multitracks dataset. Approximately 114h in total after trimming silence.
Configuration
RAVE v2, 44.1kHz, block size 2048, noncausal, Wasserstein regularization, capacity 128. Exported with 16 latent dimensions, streaming is enabled.

aam_string_b2048_r44100_z16_noncausal
Training data
All string recordings in the Artificial Audio Multitracks dataset, including viola, cello, erhu, violin, ukulele, guitar, balalaika, sitar, and jinghu. Approximately 144h in total after trimming silence.
Configuration
RAVE v2, 44.1kHz, block size 2048, noncausal, Wasserstein regularization. Exported with 16 latent dimensions, streaming is enabled.

librispeech100_b2048_r44100_z8_causal
Training data
LibriSpeech train-clean-100.tar.gz: 100 hours of 16kHz read English speech.
Configuration
RAVE v2, 44.1kHz, block size 2048, causal, variational regularization. Exported with 8 latent dimensions, streaming is enabled.

librispeech100_b2048_r44100_z8_noncausal
Training data
LibriSpeech train-clean-100.tar.gz: 100 hours of 16kHz read English speech.
Configuration
RAVE v2, 44.1kHz, block size 2048, noncausal, variational regularization. Exported with 8 latent dimensions, streaming is enabled.

aam_brass_sax_b2048_r44100_z8_noncausal
Training data
All Brass and Sax recordings in the Artificial Audio Multitracks dataset. Approximately 60h in total after trimming silence.
Configuration
RAVE v2, 44.1kHz, block size 2048, noncausal, variational regularization. Exported with 8 latent dimensions, streaming is enabled.

RAVE models migrated from nn~

The migrated folder contains several RAVE models taken from https://acids-ircam.github.io/rave_models_download, converted to neural.live~

Info

Variational and Wasserstein regularization: (taken from the tutorial on IRCAM forum) wasserstein regularization may provide better reconstruction results, at the price of a more messy latent space (no smoothness in latent exploration).

How to Cite

If you use the resources in this repository, we would appreciate citations to the following reference:

@misc{shuoyang_jasper_zheng_2025,
	author       = { Shuoyang Jasper Zheng },
	title        = { jaspers-rave-models (Revision 45e2ea8) },
	year         = 2025,
	url          = { https://huggingface.co/shuoyang-zheng/jaspers-rave-models },
	doi          = { 10.57967/hf/5589 },
	publisher    = { Hugging Face }
}

Please also make sure to attribute to the RAVE paper:

@article{Caillon_Esling_2021,
  title={RAVE: A variational autoencoder for fast and high-quality neural audio synthesis},
  url={https://doi.org/10.48550/arXiv.2111.05011},
  DOI={10.48550/arXiv.2111.05011},
  author={Caillon, Antoine and Esling, Philippe},
  year={2021} }