chatterbox-nano-coreml

Chatterbox Nano — CoreML (ANE-optimized)

On-device Apple Silicon build of ResembleAI Chatterbox Nano for iOS 18+ / macOS 15+. Prefill + decode run fully on the Apple Neural Engine as one weight-shared multifunction CoreML model; the audio synth runs as three further CoreML packages.

Links

• Upstream model — ResembleAI/chatterbox-nano by Resemble AI.
• Runtime — iliasaz/chatterbox-coreml, the Swift package + demo app that loads this repo.
• Watermarking — iliasaz/perth-coreml (Swift) / weights. Every utterance the runtime generates is watermarked with Perth, as upstream chatterbox does.

What Nano is

Nano is Chatterbox Turbo with a smaller language model — upstream's tts_nano.py is byte-identical to tts_turbo.py except for the backbone (GPT2_small vs GPT2_medium) and the T3 checkpoint. Everything downstream is the same: the S3Gen MeanFlow synth (2 CFM timesteps), the tokenizer, the conditioning, and the sampler.

Artifacts

The four .npy tables are hidden-width-specific. The Swift runtime checks them against the loaded graph at load time and refuses to run on a mismatch — pairing Turbo's 1024-wide tables with this 768-wide graph would otherwise produce garbage audio silently.

On-device numbers (iPhone 17 Pro Max, iOS 26.5)

Measured on device against a Python-CoreML-CPU baseline, same phone and harness, Turbo shown for reference:

Both are 8-bit palettized (k-means, per_grouped_channel, group 16). Nano's wpe and speech_head stay fp16 — speech_head's 6563 output channels are prime, so no group size divides them.

Mac (M5, Release CLI, same sentence): Nano generates 4.16 s of audio in 0.65 s (≈6.4× realtime); Turbo 4.00 s in 1.02 s (≈3.9× realtime). The sampler is nondeterministic, so utterance length varies run to run — read these as a rough ratio, not an exact benchmark.

Pipeline

text → BPE → host-assemble inputs_embeds + position_ids + masks
     → T3LM "prefill" function (CoreML, ANE)  — writes shared KV state
     → T3LM "decode" function (CoreML, ANE)   — autoregressive, reads/writes same state
     → S3Encoder → S3CFM → S3Vocoder (CoreML) — 24 kHz waveform

All attention masks (attn_mask, write_mask, decode update_mask) are built on the Swift host and passed in, and the prefill's attention is hand-decomposed rather than fused — on iPhone the ANE's fused attention kernel silently drops the attn_mask for q_len >> 1.

License

MIT. These weights are a CoreML format conversion of ResembleAI/chatterbox-nano, which Resemble AI publishes under the MIT licence; the conversion inherits those terms, and copyright in the weights stays with Resemble AI — no claim of ownership is made over them here. Redistribute with attribution to Resemble AI.

The Swift runtime that loads them is separately MIT-licensed; see the LICENSE and NOTICE files in iliasaz/chatterbox-coreml.