KVAE-Audio

<div align="center"> <picture> <img src="assets/kvae_audio.png"> </picture>

<a href="https://github.com/kandinskylab/kvae-audio">GitHub</a> | <a href="https://habr.com/ru/companies/sberbank/articles/1053410/">Habr article</a> | <a href="https://kandinskylab.ai/">Project Page</a> | Technical Report (soon)

</div> <h1>KVAE-Audio</h1>

KVAE-Audio is a continuous, full-band (48 kHz) audio autoencoder. It compresses raw waveforms into compact continuous latents and reconstructs them with high fidelity across speech, music, and general sound. The model is designed not only for faithful reconstruction, but as a latent space for generative models — in our internal text-to-audio pipeline, swapping the autoencoder for KVAE-Audio improves generation quality under a fixed generator.

Evaluation results

Evaluation of latent space qualities for generation

Generative quality is established under a fixed generator — same DiT architecture, training data, and number of steps — varying only the autoencoder. We report objective generation metrics and blind human side-by-side below.

<img src="assets/sbs_same_l.png" /> <img src="assets/sbs_moviegen.png" /> <img src="assets/sbs_mmaudio.png" />

AudioCaps test set

Song Describer

LibriSpeech test-clean

Reconstructions

Reconstruction is evaluated on open datasets across domains (the released weights directly substantiate these numbers). Baselines: MMAudio 44.1 kHz VAE, DACVAE from MovieGen Audio, SAME-L (Stable Audio 3 VAE).

AudioSet eval

MUSDB18-HQ

EARS