MOSS-TTS-Nano-100M-ONNX-fp16

MOSS-TTS-Nano-100M — FP16 (shared external data)

Half-precision-quantized version of OpenMOSS-Team/MOSS-TTS-Nano-100M-ONNX, re-exported with shared external weight data so prefill and decode_step mmap the same 220 MB blob instead of duplicating weights. Tuned for ARM CPUs with native fp16 SIMD (Snapdragon 8 Gen 2/3, Apple A14+, etc).

Audio codec is unchanged from upstream. Use this repo for the language model graphs and pull the codec from OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano-ONNX.

What's here

Variant comparison (this repo's role)

Triple A/B objective analysis (Adam voice, same long demo, seed=42):

Notes on fp16 numerics

The conversion uses onnxruntime.transformers.float16.convert_float_to_float16(keep_io_types=True) with these ops kept at fp32 (precision-sensitive):

• LayerNormalization
• GroupNormalization
• Softmax
• Cast

This preserves the standard "fp16 forward pass with fp32 norm/softmax" recipe that's robust across most transformer architectures. The audio output is visually indistinguishable from fp32 in spectral envelope (within the noise floor of run-to-run sampling variance — the model has stochastic decoding).

Speed expectations

If you're targeting x86 CPU, use the int8 variant instead — fp16 is a loss there.

Quick test (Python ORT)

from onnxruntime import InferenceSession, SessionOptions
from huggingface_hub import snapshot_download

lm_dir = snapshot_download("REALBITS/MOSS-TTS-Nano-100M-ONNX-fp16")
codec_dir = snapshot_download(
    "OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano-ONNX",
    allow_patterns=["moss_audio_tokenizer_decode_*", "*.json"],
)
# Drive prefill → loop {sampler → decode_step} → codec.decode_full
# (or codec.decode_step for streaming) per upstream onnx_tts_runtime.py.

On-device example

This repo is a v2 candidate for the Fictures MOSS-TTS-Nano Android prototype. Switch to it via:

adb shell setprop debug.mosstts.variant FP16

then restart the app and tap Download.

How it was quantized

Repro: see _dryrun/13_quantize_fp16.py. Three steps:

1. onnx.load(load_external_data=True) for each LM graph
2. convert_float_to_float16(keep_io_types=True, op_block_list=[LayerNorm, GroupNorm, Softmax, Cast])
3. Save with save_as_external_data=True; hash the resulting .data files; collapse byte-identical pairs (prefill + decode_step share the underlying transformer weights) into one shared moss_tts_global_shared_fp16.data and rewire both graphs' external_data location attributes.

License + attribution

Apache 2.0, inherited from upstream OpenMOSS/MOSS-TTS-Nano.

Known limitations

Same as the upstream and int8 variant: English / Chinese / Japanese only; no microphone voice cloning out of the box (use the 18 builtin voices in browser_poc_manifest.json); codec stays fp32.