magenta-rt2-mlx-quantized

Magenta RealTime 2 - Quantized MLX Weights

This repository hosts 8-bit (q8) and 4-bit (q4) quantized weights for the 2.4B base model of Google's Magenta RealTime 2 (MRT2).

Why Quantize?

The unquantized mrt2_base model uses 16-bit precision, making the weights 4.8 GB. Loading 4.8 GB of weights every 40 ms during real-time audio streaming requires ~120 GB/s of continuous memory bandwidth. On M1 Pro chips (max bandwidth 200 GB/s), this causes the weight-loading phase to take ~49 ms (123% of the audio frame budget), resulting in constant audio dropouts and stuttering.

Quantizing the model parameters reduces the weight footprint significantly, allowing real-time, jitter-free performance:

• 8-bit (q8): Compresses weights to 2.6 GB (loads in ~12–15 ms on M1 Pro). High audio fidelity, virtually indistinguishable from the 16-bit model, with zero stutters.
• 4-bit (q4): Compresses weights to 1.6 GB (loads in under 10 ms). Faster, but introduces minor audible quantization artifacts in the output.

Installation / Usage

1. Create directory structures for the quantized models under your local Magenta models folder:

   mkdir -p ~/Documents/Magenta/magenta-rt-v2/models/mrt2_base_q8
   mkdir -p ~/Documents/Magenta/magenta-rt-v2/models/mrt2_base_q4
   

2. Download and place the files:

   • For 8-bit:
     • mrt2_base_q8.mlxfn
     • mrt2_base_q8_state.safetensors inside ~/Documents/Magenta/magenta-rt-v2/models/mrt2_base_q8/
   • For 4-bit:
     • mrt2_base_q4.mlxfn
     • mrt2_base_q4_state.safetensors inside ~/Documents/Magenta/magenta-rt-v2/models/mrt2_base_q4/

3. Open your Standalone app or DAW plugin, click the MODEL dropdown in the header, and select the quantized model. Set your buffer size to 85 ms or 171 ms (buffer size 2 in the UI settings) for stable playback.