TontaubeV1

<p align="center"> <img src="assets/tontaube-logo.png" alt="Tontaube" width="180"> </p>

TontaubeV1

TontaubeV1 is a multilingual text-to-speech model designed for expressive voice cloning, long-form generation, and low-latency streaming. English and German are its primary languages; Spanish, French, Italian, Dutch, and Portuguese are also supported.

The release contains four causal codebook predictors. CB0 generates semantic audio and duration from text, while progressively smaller CB1--CB3 models add acoustic detail. Spoken text uses character-level tokenization, with compact BPE-encoded language and speaking-style controls. The public styles are audiobook, conversational, and agentic.

Technical report · Try TontaubeV1 online · Inference repository

Quick start

These checkpoints require the Tontaube vLLM model adapter and input layout; they are not drop-in AutoModelForCausalLM checkpoints. Use the matching release of the inference repository:

git clone --branch v1.0.0 --depth 1 \
  https://github.com/craitech/tontaube.git
cd tontaube
uv run --python 3.12 --frozen tontaube serve

Once the API is ready, generate a WAV with the bundled English voice from a second terminal:

curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  -d '{"text":"Welcome to Tontaube.","language":"english","tag":"conversational","format":"wav"}' \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> welcome.wav

Clone a voice by sending about 5--60 seconds of clean, single-speaker speech with the request:

VOICE_FILE=/path/to/reference.wav
python3 -c 'import base64,json,sys; print(json.dumps({"text":"This voice was cloned from a local reference.","language":"english","tag":"conversational","voice_audio_b64":base64.b64encode(open(sys.argv[1],"rb").read()).decode(),"format":"wav"}))' "$VOICE_FILE" \
| curl --silent --show-error --fail http://127.0.0.1:8080/predict \
  -H 'content-type: application/json' \
  --data-binary @- \
| python3 -c 'import base64,json,sys; sys.stdout.buffer.write(base64.b64decode(json.load(sys.stdin)["audio_b64"]))' \
> cloned.wav

The server downloads this model and its pinned runtime dependencies into local caches on first use. Follow the inference repository's installation instructions for system requirements, the browser interface, and Docker. The optional English verbalizer is available separately from TontaubeAI/TontaubeV1-Verbalizer and is downloaded only when enabled.

The default semantic sampling temperature is 0.8. A value around 0.6 generally favors text correctness, while values approaching 1.0 can produce more varied and expressive delivery.

Capabilities

• Bundled synthetic voices and zero-shot cloning from up to 60 seconds of reference audio
• audiobook, conversational, and agentic speaking styles
• Bounded-context long-form generation
• WAV generation and low-latency MP3 or Opus streaming
• Optional English text verbalizer, distributed separately

Examples

These MP3 samples were normalized to a consistent speech listening level.

Audiobook

English 1

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/english-audiobook-01.mp3"></audio>

Download MP3

English 2

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/english-audiobook-02.mp3"></audio>

Download MP3

English 3

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/english-audiobook-03.mp3"></audio>

Download MP3

German 1

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/german-audiobook-01.mp3"></audio>

Download MP3

Agentic

English 1

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/english-agentic-01.mp3"></audio>

Download MP3

English 2

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/english-agentic-02.mp3"></audio>

Download MP3

English 3

<audio controls preload="none" src="https://huggingface.co/TontaubeAI/TontaubeV1/resolve/main/assets/showcase/english-agentic-03.mp3"></audio>

Download MP3

Model architecture

CB0 uses a Qwen3-1.7B-derived transformer trunk. CB1--CB3 use progressively shallower Qwen3-0.6B-derived trunks. Each predictor replaces the inherited language-model output layer with a two-layer audio-token head. The four output streams are decoded with DualCodec; the release inference path then uses the acoustic encoder and decoder from VibeVoice for continuous reconstruction and streaming.

Training

All four predictors were trained exclusively with supervised fine-tuning on approximately 200,000 hours of paired speech and text across seven languages, predominantly from public-domain audiobook recordings and openly released speech corpora.

Repository contents

TontaubeV1/
├── cb0/
├── cb1/
├── cb2/
└── cb3/

Each directory contains its safetensors weights, runtime configuration, and tokenizer files. The repository does not redistribute DualCodec, W2V-BERT, or VibeVoice weights. The inference setup downloads those components from their upstream repositories at pinned revisions.

Serving performance

With weights resident on one NVIDIA GeForce RTX 5090 and the process warmed, the streaming path reaches approximately 200 ms to first encoded audio. In separate non-streaming measurements, end-to-end real-time factor (RTF) is 0.08 for one input and aggregate RTF is 0.02 across eight concurrent inputs. Startup, model loading, and network latency are excluded.

Evaluation

The LLM-as-a-judge pairwise audiobook-reading benchmark contains 400 fixed English passages of 250--500 characters. For each passage, Gemini 3.1 Pro Preview judges the same output pair twice, once in each presentation order, on prosody and word-by-word correctness. A TontaubeV1 preference, tie, or comparator preference scores 1, 0.5, or 0; the figure reports the mean over all 800 order-balanced judgments. A score of 50% denotes parity. Whiskers show 95% passage-cluster bootstrap intervals; labels beneath the points give tie rates across individual judge calls.

<p align="center"> <img src="assets/llm-judge-results.png" alt="TontaubeV1 preference scores against ElevenLabs Flash v2.5, Fish Audio S2 Pro, Gradium, and Cartesia on prosody and correctness."> </p> <details> <summary>Exact preference scores</summary> </details>

TontaubeV1 uses semantic sampling temperature 0.55 and acoustic temperature zero in these comparisons. Each waveform is independently normalized to an average level of -20 dBFS before judging. The technical report gives the complete judge instructions and further methodological limitations.

Fish Audio uses the same frozen cloning reference as TontaubeV1. ElevenLabs, Gradium, and Cartesia instead use fixed provider voices while TontaubeV1 uses the cloning reference. Although voice identity and timbre are excluded from the rubric, prosody is not fully separable from the reference; this asymmetry may favor TontaubeV1 in those three comparisons.

On the 1,088 English zero-shot examples of the Seed-TTS evaluation set, TontaubeV1 obtains 1.66% mean utterance-level word error rate using Whisper large-v3 transcription at semantic sampling temperature 0.6.

These automated evaluations measure English reading prosody and text correctness. They do not establish voice similarity, overall sound quality, multilingual quality, long-form continuity, or streaming quality.

Limitations and responsible use

Autoregressive generation can omit, repeat, or alter text and can terminate too early or too late. Reference conditioning may reproduce incidental recording properties or imperfectly preserve identity. Quality varies by language, speaker, style, and text domain. The optional verbalizer can normalize incorrectly or alter wording. Training emphasized audiobook speech, so audiobook generation may be more reliable than conversational or agentic generation.

English and German are the primary languages. German phoneme realization can be imperfect, and Spanish, French, Italian, Dutch, and Portuguese have not been evaluated by native speakers; no comparative quality claims are made for those languages.

Voice cloning can enable impersonation, fraud, non-consensual synthesis, and misleading media. Obtain permission for reference voices and apply appropriate authentication, rate limiting, disclosure, logging, and abuse-response measures in deployed systems. Generated speech must not be treated as verified identity or factual evidence.

License and attribution

TontaubeV1 is distributed under the Tontaube Community Model License 1.0, which is not an open-source license. The license applies to the Tontaube model materials identified by the release manifest; third-party material remains subject to its respective terms. See Third-Party Notices.

TontaubeV1 builds on Qwen3 and uses the DualCodec speech representation. The release inference stack also uses components from VibeVoice, W2V-BERT, PyTorch, Transformers, and vLLM. Their respective licenses and notices remain applicable.

The generated audio examples in assets/showcase/ are provided as demonstration material and are not part of the model materials covered by the Tontaube Community Model License 1.0.