Audio8-TTS-Preview-0.1b-bf16

Audio8-TTS-Preview-0.1b-bf16 (MLX)

MLX conversion of Audio8/Audio8-TTS-Preview-0.1b (revision b476f02) for Apple Silicon — a ~170M-parameter multilingual text-to-speech model with zero-shot voice cloning and a bundled 44.1 kHz neural codec.

• Language model: bf16 (as published upstream), 170M parameters
• Codec: fp32, weight-norm folded and conv weights in MLX (channels-last) layout
• Architecture: arktts with a Falcon-H1 hybrid slow stack — every slow layer carries a Mamba-2 mixer, attention, and an MLP. A 4-layer fast AR predicts the 10 codec codebooks per frame, as in the 0.6b.

⚠️ Requires mlx-audio with slow_backbone: falcon_h1 support

This checkpoint sets slow_backbone: falcon_h1, which the released arktts model (mlx-audio ≥ 0.4.7) does not yet understand — that release only handles the 0.6b's pure-attention slow stack. Until the follow-up lands in a release, install from source:

pip install "git+https://github.com/Blaizzy/mlx-audio@main"

If your mlx-audio predates falcon_h1 support, loading this repo fails at weight loading (the key layouts differ); it does not silently produce bad audio.

Usage (mlx-audio)

import soundfile as sf
from mlx_audio.tts.utils import load

model = load("mlx-community/Audio8-TTS-Preview-0.1b-bf16")

# zero-shot voice cloning: the reference transcript must match the reference audio
for result in model.generate(
    text="Welcome to Audio8 TTS, running on Apple Silicon.",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference clip.",
    temperature=0.7, top_p=0.9, top_k=50,
):
    sf.write("output.wav", result.audio, result.sample_rate)

Without ref_audio, the model synthesizes with a default voice.

Conversion parity

Verified against the PyTorch reference (fp32, CPU, transformers 4.57.6):

A note for anyone re-implementing this

embedding_multiplier (0.10888671875) applies to the composite slow-AR input — text embedding plus the ten codebook embeddings — not to the token lookup alone. The weights in this repo therefore ship with a raw embed_tokens table, unlike a plain Falcon-H1 MLX conversion where the multiplier is folded into it.

Folding it here scales the text half and leaves the codebook half at 1.0: measured against the reference that is 77% relative error on the embedding and 38% on the final hidden state — while still producing plausible-sounding audio of the correct length, so it will not be caught by listening. Either apply the multiplier to the sum, or fold it into the table and scale the codebook sum by it as well; the two are algebraically identical.

Supported languages

Primary: Chinese and English. Experimental: German, Spanish, French, Italian, Japanese, Korean. Per the upstream model card, non-primary languages are weaker and more variable than in the 0.6b.

Choosing between this and the 0.6b

mlx-community/Audio8-TTS-Preview-0.6b-bf16 remains the higher-quality option, particularly for speaker similarity, which is the metric that matters most for zero-shot cloning (upstream Seed-TTS SIM: 63.2 EN / 73.1 ZH for the 0.6b vs 56.7 / 68.2 here). This 0.1b is the choice when download size matters: 1.7 GB total against 2.6 GB.

It is not a large memory win. Both share the same 1.35 GB fp32 codec, which dominates resident footprint and peak activation. Measured back-to-back on the same text and reference clip, deterministic decoding, warm-up run discarded:

A 3.5x smaller language model buys roughly 15% throughput and 7% peak memory, because the codec is unchanged.

License

The language model is released under the Audio8 Community License v1.0, a revenue-capped custom license: free for non-commercial use, free for commercial use by entities under US$2,000,000 annual revenue, and requiring a separate written license from Audio8 at or above that threshold. Review it before deploying.

The bundled codec weights are Apache-2.0. Upstream ships a byte-identical codec in the Apache-2.0 Audio8-TTS-Preview-0.6b repository (verified: same file checksum, and bit-identical outputs across the encoder, quantizer decode, and full decode); the tensors here were converted from that repository.