neutts-jp-150m-onnx-int8

neuTTS-JP-150M — ONNX LM (weight-only INT8)

This repository hosts the weight-only INT8 ONNX export of the causal language model used in the Japanese neuTTS-JP-150M stack (LM + NeuCodec). It is a companion artifact: tokenizer, config.json, and the FP16 baseline ONNX live in the float16 bundle repository.

What is here

Approximate size on disk: ~152 MB (vs ~304 MB FP16 in the companion repo).

Companion repository (required for a full app layout)

Download tokenizer and config (and FP16 fallback if you want) from:

aoiandroid/neutts-jp-150m-onnx-float16

Typical local layout after merging both sources:

• From float16 repo: tokenizer.json, tokenizer_config.json, config.json, model_fp16.onnx (optional fallback), and any other files shipped there.
• From this repo: model_int8.onnx — place it in the same directory as the tokenizer if your client expects a single folder.

on-device iOS client resolves the LM file using the candidate order model_int8.onnx then model_fp16.onnx when both exist under the configured model directory (see NeuTTSJP150MONNXService.onnxModelCandidateFileNames in the app sources).

Quantization

• Tooling: onnxruntime quantization — dynamic weight-only path (QuantType.QInt8).
• Script reference (local project): Scripts/neutts_jp_150m_onnx/quantize_weight_only.py — input model_fp16.onnx, output model_int8.onnx.
• Not included: static / activation calibration quantization; if you need that, use a separate calibrated pipeline.

Re-quantize with an ONNX Runtime version close to the one you ship on-device (iOS/macOS xcframework) to reduce kernel or accuracy surprises.

Inference contract

The INT8 graph is intended to be API-compatible with model_fp16.onnx from the float16 repo (same inputs/outputs and past_key_values layout). Empty KV cache tensor element types should follow the loaded model metadata (the app bridge reads ORT types from the loaded session).

License and upstream

Licensing and model lineage follow the upstream neuTTS-JP / float16 Hub documentation. Use the model card and LICENSE (if any) on aoiandroid/neutts-jp-150m-onnx-float16 and the base checkpoints they reference.

Download examples

# This repo (INT8 LM only)
hf download aoiandroid/neutts-jp-150m-onnx-int8 --local-dir ./neutts-jp-onnx-lm-int8

# Companion (tokenizer + FP16 + config)
hf download aoiandroid/neutts-jp-150m-onnx-float16 --local-dir ./neutts-jp-onnx-float16

Merge model_int8.onnx into your runtime directory next to the tokenizer files from the float16 snapshot if your loader expects one folder.

Disclaimer

This artifact is provided as-is for on-device experimentation. Validate audio quality and stability on your target hardware before production use.