magenta-rt-2-qad-onnx

Magenta RT 2 — Quantized Acoustic Distillation (QAD) ONNX

This repository hosts the Quantized Acoustic Distillation (QAD) ONNX models for real-time neural music generation in the browser via WebGPU and WASM SIMD.

Architecture & Model Components

1. authentic_student_generator_int8.onnx (64.0 MB):

   • Unrolled 12-codebook Depthformer student model (6 temporal layers, 2 depth layers, 35.7M active parameters).
   • Generates 12 hierarchical Residual Vector Quantizer (RVQ) acoustic codes causally per 40 ms frame in ~39 ms on CPU (~12 ms on WebGPU).
   • Conditioned on 768-dim MusicCoCa style embeddings and 128-element MIDI chord/note vectors.

2. authentic_spectrostream_decoder_int8.onnx (41.9 MB):

   • Full 35.6M-parameter SpectroStream neural vocoder.
   • Converts 12 RVQ codes into 48 kHz stereo PCM audio via convolutional frame expansion, harmonic synthesis, and WebGPU/WASM-native MatMul IDFT overlap-add.

3. style_embeddings.json (144 KB):

   • Precomputed 768-dimensional latent style vectors across curated genres: Lo-Fi, Chill-Hop, Smooth Jazz, Synthwave, Deep Ambient, Classical, Rock, Funk, and Cinematic.

Direct CDN & Download Links

• authentic_student_generator_int8.onnx: https://huggingface.co/JustACluelessKidAtSchool/magenta-rt-2-qad-onnx/resolve/main/authentic_student_generator_int8.onnx
• authentic_spectrostream_decoder_int8.onnx: https://huggingface.co/JustACluelessKidAtSchool/magenta-rt-2-qad-onnx/resolve/main/authentic_spectrostream_decoder_int8.onnx
• style_embeddings.json: https://huggingface.co/JustACluelessKidAtSchool/magenta-rt-2-qad-onnx/resolve/main/style_embeddings.json

In-Browser Deployment

Designed for 100% client-side zero-backend browser execution in:

• music_studio.html: Interactive live jam studio with prompt latent surface and keyboard steering.
• focus_radio.html: Continuous endless AI radio stream with ambient soundscape mixer.
• Models are cached into browser IndexedDB (CascadeOfflineMusicDB) on first download for permanent offline execution.