stable-audio-open-small-onnx

Stable Audio Open Small — ONNX export

Powered by Stability AI

An ONNX export of stabilityai/stable-audio-open-small, packaged so it runs through ONNX Runtime (WebGPU or CPU) with no Python or PyTorch install.

It exists because the weights are too large to ship inside a desktop installer. It is used by QuickCut, a video editor that generates short sound effects from a text prompt entirely on the local machine, which downloads these files once on first use.

The weights are unchanged — this is a format conversion, not a retrain. See NOTICE for exactly what was modified.

Files

Total ≈ 2.0 GiB. All graphs are fp32.

Model

341M-parameter DiT + 156M Oobleck autoencoder + t5-base text encoder. Up to 11.89 s of 44.1 kHz stereo audio. This is the distilled variant: 8-step ping-pong sampling at CFG 1, one DiT evaluation per step.

The three graphs correspond to the three stages of a generation:

prompt ──> tokenizer.json  (t5-base Unigram, Viterbi)
       ──> cond.onnx        text + numeric conditioning
       ──> dit.onnx         x8  ping-pong denoising steps
       ──> dec.onnx         latent -> 44.1 kHz stereo waveform

Sampling schedule used by the reference implementation, for anyone reproducing it: linspace(1 -> 0, 9) then sigmoid(8.2t - 2.0) (LogSNRShift rate 0), endpoints preserved, t0 = 1. Each step is den = x - t * DiT(x, t, cond) then x = (1 - t_next) * den + t_next * noise.

dec.onnx has a dynamic sequence axis, so a 2-second request decodes at roughly a sixth of the cost of a full-length one.

A note on cond.onnx

On the hardware this was built for, the conditioning graph is dramatically faster on CPU than on WebGPU (29 ms vs 851 ms) — it is small and launch-latency dominated. Running cond on CPU while dit and dec run on WebGPU is worth trying if you are chasing latency.

Reproducing the export

1. Get model.safetensors and model_config.json for Stable Audio Open Small.
2. git clone --depth 1 Stability-AI/stable-audio-tools (verified against commit 3241adb).
3. Install torch (CPU is fine) plus einops einops-exts alias-free-torch transformers sentencepiece safetensors scipy PyWavelets torchsde torchaudio numpy onnx.
4. Run export_onnx.py (included here), which already handles the three non-obvious parts: tracing the decoder at a short length with a dynamic sequence axis, disabling model_half so no fp16 casts get baked in, and removing weight norm before export.
5. tokenizer.json comes from google-t5/t5-base.

License

The model weights are licensed under the Stability AI Community License — free for research, non-commercial, and commercial use by anyone generating under USD $1,000,000 in annual revenue. A copy is included as LICENSE, and NOTICE records the required attribution and the modifications made.

If you use this for a commercial purpose, the license asks you to register at https://stability.ai/community-license. Above the revenue threshold you need an enterprise license from Stability AI.

tokenizer.json is from google-t5/t5-base and is Apache-2.0, not a Stability AI Material.

The original repository is gated; please also review the terms there.