Fish-Audio-S2-Pro-NVFP4-Balanced

<h1 align="center">V1 · Fish Audio S2-Pro · NVFP4 Balanced</h1> <p align="center"><strong>A complete Blackwell-ready S2-Pro download:</strong> mixed native NVFP4/MXFP8 transformer weights, BF16 codec, tokenizer, pinned Fish Speech source, API server, web UI, and reproducibility checks.</p> <p align="center"><strong>Balanced V1 prioritizes useful English quantization fidelity and VRAM reduction. This is not yet an XPO3 release.</strong></p> <p align="center"><strong>An XPO3 version is coming soon.</strong> Follow <a href="https://huggingface.co/ajh-code">ajh-code on Hugging Face</a> and <a href="https://arands.com">Arands.com</a> for release updates.</p> <p align="center"> <a href="https://huggingface.co/fishaudio/s2-pro">Original S2-Pro</a> · <a href="https://github.com/fishaudio/fish-speech">Fish Speech</a> · <a href="https://arands.com">Arands.com · updates</a> </p>

---

Download

All model and codec weights required by the server are in this repository. The root config.json preserves the S2-Pro architecture metadata and adds the mixed-precision policy, while Hugging Face metadata records this repository as a quantization of fishaudio/s2-pro.

Quick start

Tested on Linux x86-64, Python 3.12, CUDA 13.0, PyTorch 2.11.0+cu130, comfy-kitchen==0.2.22, and NVIDIA Blackwell SM120. The current native path is for GeForce RTX 50-series/SM120 GPUs; it is not a generic CUDA fallback.

hf download ajh-code/Fish-Audio-S2-Pro-NVFP4-Balanced \
  --local-dir fish-audio-s2-pro-nvfp4-balanced
cd fish-audio-s2-pro-nvfp4-balanced
./install.sh
./launch.sh

Open http://127.0.0.1:8080/ui for the bundled zero-shot web interface. The API listens on all interfaces by default; set TTS_HOST=127.0.0.1 if it should not be reachable from the local network. Protect or firewall the service before exposing it beyond a trusted network.

Docker Compose is the recommended clean deployment when Docker, the NVIDIA Container Toolkit, and a compatible driver are already configured:

docker compose up --build

This path passed a clean outer-Docker build, SM120 runtime launch, route/UI checks, and a zero-shot API smoke test on an RTX 5080 with CUDA 13.0.

The image excludes the 6.8 GB model payload and mounts the downloaded repository read-only, so rebuilding the runtime does not duplicate the weights inside the image.

Zero-shot voice cloning

Use a clean, consented 10–30 second reference with one speaker and supply its exact transcript:

python client.py \
  --url http://127.0.0.1:8080/v1/tts \
  --reference-audio reference.wav \
  --reference-text "The exact words spoken in reference.wav." \
  --text "A few notes as this story begins." \
  --seed 42 \
  --output result.wav

Equivalent JSON API call in Python:

import base64
from pathlib import Path

import requests

payload = {
    "text": "A few notes as this story begins.",
    "references": [{
        "audio": base64.b64encode(Path("reference.wav").read_bytes()).decode(),
        "text": "The exact words spoken in reference.wav.",
    }],
    "reference_id": None,
    "format": "wav",
    "streaming": False,
    "normalize": True,
    "max_new_tokens": 1024,
    "chunk_length": 200,
    "top_p": 0.9,
    "temperature": 0.9,
    "repetition_penalty": 1.1,
    "seed": 42,
    "use_memory_cache": "off",
}
response = requests.post("http://127.0.0.1:8080/v1/tts", json=payload, timeout=600)
response.raise_for_status()
Path("result.wav").write_bytes(response.content)

Useful endpoints:

Set TTS_API_KEY before launch to require bearer authentication. For an API key named secret, send Authorization: Bearer secret.

Quantization policy

S2-Pro has 180 projections in its 36-layer slow transformer. V1 uses:

The English calibration scale is folded into the selected norm and packed gate/up tensors. It adds no runtime tensor or operation. The original BF16 slow-projection weights are not retained as a second copy. V1's transformer checkpoint is 46.25% smaller than the original transformer shards.

This is intentionally described as a mixed NVFP4/MXFP8 checkpoint. It is not a claim that every operation, activation, or weight in the end-to-end TTS stack runs at FP4.

Measured performance

Measurements below are local RTX 5080 results with the bundled compact BF16 codec path and a 3072-token cache. They are not universal performance claims.

The matching compact-runtime BF16 control loaded at 9.273 GiB, so V1 reduced loaded PyTorch allocation by 42.31%. V1 is near real time on the RTX 5080, but the current ordinary single-speaker API waits for a complete semantic segment before playable audio. This release therefore does not claim agent-grade low-latency streaming. An RTX 5060 Ti focused zero-shot run measured about 2.06 RTF and is not a real-time path.

Limited blind voice-cloning test

The first blind comparison is encouraging, but deliberately small. It used one listener, one consented English reference speaker, and four matched BF16/V1 pairs: conversational, reflective, question-shaped, and long narrative prompts at seeds 7, 17, 123, and 42. Both models used the same reference, runtime path, temperature=0.9, top_p=0.9, and top_k=30.

The fourth pair was tied. The only severe artifact reported in the set was a deterministic BF16 pitch squeak in the long seed-42 sample; its V1 counterpart did not contain that excursion.

This test suggests that the quant did not cause a detectable speaker-identity loss for that reference. It is not a general MOS study or broad cloning qualification: more listeners, speakers, accents, recording conditions, and languages are still needed. The release therefore reports the result without claiming parity in every voice-cloning setting.

Validated scope

The blind result supports quantization fidelity for that English reference; it does not establish universal cloning quality across voices, recording conditions, accents, or languages.

Known limitations

• Fish S2-Pro itself sounded substantially flatter and less expressive than VoxCPM2 in our reference comparison. BF16 shared this behavior, so V1 does not treat it as NVFP4-specific damage and does not claim to fix it.
• Inline emotion/style instructions change output trajectories but did not reliably repair the perceived flatness in the tested voice.
• The objective speaker embedding saturated near 0.99 and failed to predict human preference; human listening remains required for new voices.
• One matched long BF16 sample produced a deterministic pitch squeak while its V1 counterpart did not. This is evidence from one seed, not a claim that V1 is generally more artifact-free than BF16.
• Long-form peak memory is materially higher than loaded memory. Do not market V1 as a sub-6-GiB operational model for arbitrary request lengths.

These bounded claims are why this package is Balanced V1, not an XPO3 speed/quality/size release. Follow ajh-code for the upcoming XPO3 version.

Validate the download

python validate_release.py

MANIFEST.json records the byte size and SHA-256 of every distributed file except itself. Validation also checks the safetensors index/header mapping, the 60/120 NVFP4/MXFP8 tensor counts, source pins, license/notice files, and runtime payload. Hashing the 6.8 GB package takes a little while.

For an additional hash check every time the service loads:

TTS_VERIFY_CHECKSUMS=1 ./launch.sh

License and attribution

Built with Fish Audio. This derivative is governed by the Fish Audio Research License. Research and non-commercial use are permitted subject to its terms. Commercial use requires a separate written license from Fish Audio; no commercial rights are granted by this repository. See Notice for the required attribution and exact change statement, and THIRD_PARTY_NOTICES.md for runtime dependencies.

Use only voices and recordings you have the right and consent to use.