stable-audio-open-small

Stable Audio Open Small — Mobile Optimized (ONNX)

This repository hosts mobile-optimized runtime artifacts and converted weights for Stable Audio Open Small (341M parameters), adapted for on-device generative audio synthesis in the Neural Beats mobile app (iOS & Android, via ONNX Runtime Mobile).

---

Model Architecture & Overview

• Base Architecture: Stable Audio Open Small (Stability AI / Arm), split into 3 independently exported ONNX graphs run in sequence:
  • T5 text encoder (t5_text_encoder.onnx) — encodes the text prompt into cross-attention conditioning embeddings.
  • DiT step model (dit_step.onnx) — a single denoising step of the diffusion transformer; invoked repeatedly (8 rectified-flow pingpong steps) by the on-device sampler.
  • Oobleck VAE decoder (vae_decoder.onnx) — decodes the 64-channel latent produced by the sampler into 44.1 kHz stereo audio (2048× upsampling).
• Segment Output: Generates ~10.0-second seamless stereo audio passes at 44.1 kHz (215 latent steps × 2048 downsampling ratio ÷ 44.1 kHz).

---

Weight Precision (INT8 storage)

All three ONNX models use weight-only INT8 storage: every weight tensor above a small-tensor threshold is symmetrically quantized to int8 (per-output-channel scale for 2D+ weight matrices, per-tensor scale for 1D tensors), with an explicit DequantizeLinear(int8→fp32) op inserted ahead of each tensor's original consumer(s). The compute graph itself is untouched — every op still runs in float32 exactly as before, and both model inputs/outputs remain float32 — so there is no runtime behavior change for native iOS/Android inference code, only a much smaller download (roughly a quarter of the original FP32 size, half of the previous FP16-storage size). Quantization introduces more noise than FP16 storage but remains well within an acceptable range for ambient/generative audio use cases (validated via onnxruntime: no clipping, no NaNs, >0.98 waveform correlation vs. the FP16-storage version on identical inputs).

Each .onnx file is a small graph-structure file; the bulk of each model's weight data lives in a companion .onnx.data external-data file with the same base name (ONNX Runtime resolves this automatically as long as both files are downloaded into the same directory).

---

Hosted Files & Formats

Total download size: ~0.5 GB (down from ~1.0 GB at FP16-storage precision, ~2.0 GB at full FP32 precision).

---

License & Attribution

This model is derived from Stability AI's Stable Audio Open Small and is distributed under the Stability AI Community License.