Roo-Voice_MOSS_TTS_LT_int4

Roo-Voice · MOSS-TTS-Local-Transformer · INT4 (bitsandbytes NF4 / CUDA)

Roo's voice — that signature baritone with the estuary accent that stands the hair up on the back of your neck — as a 4-bit checkpoint that runs on a modest NVIDIA GPU. Load it, and Roo can whisper to you all day long.

An INT4 (NF4, bitsandbytes) quantization of a full-model supervised fine-tune of MOSS-TTS-Local-Transformer, trained on Roo's own recordings and specialised on his single voice. This is the lightest-VRAM transformers / NVIDIA-GPU form — for full precision use the BF16 build, for a bigger quality headroom use the INT8 build, and for Apple Silicon use the MLX 4-bit build.

⚠️ What actually makes the voice — read first

This is a reference-conditioned model, and both halves matter:

• The fine-tune is what makes it Roo. The base model has never heard this speaker — a base model plus any reference clip will not give you Roo's baritone or his estuary accent. That voice lives in the weights, put there by the supervised fine-tune on his recordings.
• The reference completes the delivery. reference.wav (bundled) conditions the fine-tuned model at inference and is required to produce the voice.

So the product is this fine-tune + reference.wav, together — not a text-only model, and not a generic voice-cloner (swap the reference and it isn't Roo, because the accent and timbre are the fine-tune's, not the clip's).

What it is

Because the voice-sensitive heads/embeddings stay in bf16 (they carry the timbre and set the size floor), NF4 lands around 3.5 GB — smaller than INT8, not a full halving. If you want the smallest file, or a single 4-bit that also runs on AMD / CPU, use the Q4_K_M GGUF via llama.cpp (~1.9 GB, Vulkan / ROCm / CUDA).

Requirements

pip install "transformers==5.0.0" bitsandbytes accelerate torch soundfile

• A CUDA NVIDIA GPU (bitsandbytes 4-bit requires CUDA).
• transformers==5.0.0 to match the model's remote code.
• The quantization config is embedded — from_pretrained loads it in NF4 4-bit automatically. The backbone cache dimensions are mirrored onto the top-level config so generate() works out of the box.

Usage

This is a drop-in fine-tuned MossTTSLocal checkpoint. Run it via the standard MOSS-TTS Local inference path (PyTorch/CUDA) and pass reference.wav as the reference:

• OpenMOSS MOSS-TTS reference pipeline — https://github.com/OpenMOSS/MOSS-TTS.
• AutoProcessor + AutoModel (trust_remote_code=True) → build_user_message(text=..., reference=[...]) → model.generate(...) → processor.decode(...).

Decoding contract for this voice: seed 42, temperature 1.0, top-k 50, top-p 0.95, repetition penalty 1.1, 32 RVQ codebooks.

Limitations

• Reference-conditioned — the bundled reference.wav must ride along; there is no text-only path.
• Single voice by design (this is Roo, not a multi-speaker system).
• NF4 4-bit backbone quantization: a slightly larger quality delta vs BF16/FP32 than INT8 is possible; the sensitive heads and embeddings are kept bf16 to preserve timbre.

Provenance & license

Quantized/exported form of an accepted single-speaker MOSS-TTS Local supervised fine-tune. The base model and audio codec are Apache-2.0 (OpenMOSS); weights derived from them are redistributed here under the same license.