Qwen3-TTS-12Hz-1.7B-NVFP4

Qwen3-TTS-12Hz-1.7B-CustomVoice - NVFP4A16 Quantized

Model Overview

This repository contains a pre-quantized NVFP4A16 derivative of Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice.

It is a weight-only post-training quantization of the model's autoregressive speech-generation decoder, stored in the compressed-tensors format. The rest of the model remains in BF16.

Qwen3-TTS CustomVoice model supports:

• 10 languages: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian
• 9 preset voices in the CustomVoice variant
• instruction-aware style control

This derivative is intended for users who want a smaller, faster checkpoint while staying close to the original model's behavior.

Quantization Summary

• Base model: Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
• Quantization type: NVFP4A16, weight-only
• Storage format: compressed-tensors
• Quantized scope: autoregressive decoder layers (talker.model.layers.*)
• Kept in BF16: speech tokenizer, embeddings, text projection, codec/output layers, and other non-decoder components

Approximate repository size: 2.4 GB

Practical Gains

Compared with the BF16 base model, this checkpoint delivered:

• 1.4x faster generation at median RTF (30.6% lower)
• 1.3x lower first-audio latency (24.8% lower)
• 2.1x lower decoder weight memory (52.8% lower)

Evaluation

Benchmark setup

The benchmark numbers below come from local measurements collected on an NVIDIA DGX Spark system.

• Corpus size: 100 fixed multilingual prompts
• Language distribution: Japanese 24, English 20, French 14, Chinese 14, German 6, Spanish 6, Italian 6, Korean 6, Russian 2, Portuguese 2
• Latency metrics: real-time factor (RTF) for standard request generation; first-audio latency and uninterrupted-playback start delay for streaming generation
• Quality metric: character error rate (CER) computed by transcribing generated audio back with Qwen3-ASR-1.7B

Standard request generation

Streaming generation

Relative to the BF16 baseline, this checkpoint measured:

• 30.6% lower median RTF for standard requests and streaming requests
• 24.8% lower median first-audio latency
• 35.1% lower median gapless-start delay
• no median CER degradation on the saved benchmark set

Repository Contents

• model.safetensors - quantized checkpoint
• config.json - model config including quantization metadata
• generation_config.json - generation defaults
• preprocessor_config.json - preprocessing metadata
• tokenizer_config.json, vocab.json, merges.txt - tokenizer assets
• speech_tokenizer/ - speech tokenizer assets required by the upstream architecture
• recipe.yaml - quantization scope metadata

Compatibility Notes

This repository contains weights only. Use software that already supports:

• the upstream Qwen3-TTS architecture
• pre-quantized compressed-tensors checkpoints
• NVFP4-style decoder quantization

If you are evaluating software stacks, a recent release of vLLM / vLLM-Omni with audio-model and compressed-tensors support is a reasonable starting point.

Sources

• Original model: Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
• Upstream technical report: Qwen3-TTS Technical Report (arXiv:2601.15621)
• Quantization tooling: llm-compressor
• Quantized checkpoint format: compressed-tensors

License Reminder

This repository is a derivative of an upstream model released under Apache-2.0.

Please review and comply with the original model license, notices, and any upstream usage guidance before redistributing, serving, or incorporating this checkpoint into another product or repository.