Downloads · 30 days
0
harvestsu/qwen3-tts-0.6b-base-jetson-trtllm-int4fp8
qwen3-tts-0.6b-base-jetson-trtllm-int4fp8 is a machine learning model from harvestsu. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Streaming, real-time TTS for the Qwen3-TTS-12Hz-0.6B-Base model on NVIDIA Jetson Orin (sm87, e.g. Orin Nano 8GB), built on TensorRT-Edge-LLM v0.8.0.
Downloads · 30 days
0
Access
Public
Updated Jun 20, 2026
Repo size
871 MB
Likes
0
Public
Click a slice to open those files.
.tgz871 MB · 100%
From the Hugging Face model README
Streaming, real-time TTS for the Qwen3-TTS-12Hz-0.6B-Base model on NVIDIA Jetson Orin (sm_87, e.g. Orin Nano 8GB), built on TensorRT-Edge-LLM v0.8.0.
Optimized, fully upstream-native quantization (no fork plugin backports):
jv-workers/
qwen3_tts_streaming_worker # JSON-line streaming worker (built ENABLE_CUTE_DSL=OFF)
libNvInfer_edgellm_plugin.so # TRT plugins (AttentionPlugin + Int4GroupwiseGemmPlugin)
models/qwen3-tts-base/
engines/talker/ # int4 talker engine + fp8 text_embedding.safetensors (table+scales) + sidecars
engines/code_predictor/ # int4 CodePredictor engine + codec_embeddings + lm_heads
engines/code2wav/ # fp16 vocoder engine (maxCodeLen=128)
ref_embedding.b64.txt # fixed speaker embedding (ref_06 5.9s, loudness-normalized), base64 of 1024-d f32
export EDGELLM_PLUGIN_PATH=$PWD/jv-workers/libNvInfer_edgellm_plugin.so
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
M=$PWD/models/qwen3-tts-base
./jv-workers/qwen3_tts_streaming_worker \
--talkerEngineDir=$M/engines/talker \
--codePredictorEngineDir=$M/engines/code_predictor \
--code2wavEngineDir=$M/engines/code2wav \
--tokenizerDir=$M/engines/talker
# then write JSON request lines to stdin, e.g.:
# {"id":"t","text":"Hello world.","speaker_embedding_b64":"<contents of ref_embedding.b64.txt>",
# "stream":true,"chunk_frames":13,"first_chunk_frames":4,"talker_temperature":0.9,
# "chunk_format":"pcm_s16le","chunk_transport":"base64"}
# events: ready / chunk{audio_b64} / done
Notes: talker_temperature 0.9 (lowering breaks EOS); first_chunk_frames 4 for low TTFA. The fixed voice = the precomputed ref_embedding.b64.txt; replace it to change voice (compute via the base model's speaker_encoder on a clean ≥3s clip, loudness-normalized).
github.com/suharvest/TensorRT-Edge-LLM branch port/qwen3-tts-base-v080.wip/native-int4-talker (quantize_talker_stage1.py, stage2_export.py, quantize_cp_stage1.py, cp_stage2_export.py) — rebuild as transformers.Qwen3ForCausalLM → native INT4_AWQ_CFG → re-prefix → existing _export_*; exclude lm_head (+ CP down_proj fp32) for EOS.scripts/quantize_text_embedding_fp8.py (e4m3, per-group 128, scale=amax/448).llm_build for talker/CP, audio_build --maxCodeLen 128 for code2wav, with ENABLE_CUTE_DSL=OFF (CuTe-DSL GEMM has no sm_87 SASS) and EDGELLM_PLUGIN_PATH set.Qwen/Qwen3-TTS-12Hz-0.6B-Base.