Kitsune-TTS-V1

<p align="center"> <a href="https://github.com/Heitorkk2/Kitsune-TTS"> <img src="https://raw.githubusercontent.com/Heitorkk2/Kitsune-TTS/main/assets/logo.png" alt="Kitsune-TTS Logo" width="480"/> </a> </p> <p align="center"> <em>Ultra-lightweight Text-to-Speech for anime personas and virtual assistants.</em> </p> <p align="center"> <strong>Brazilian Portuguese</strong> 🇧🇷 &nbsp; · &nbsp; ~39M parameters &nbsp; · &nbsp; 5 voices </p> <p align="center"> <a href="https://huggingface.co/Heitorkk2/Kitsune-TTS-V1"> <img src="https://img.shields.io/badge/Hugging%20Face-Model-gray?logo=huggingface" alt="Hugging Face: Kitsune-TTS-V1"/> </a> <a href="https://github.com/Heitorkk2/Kitsune-TTS"> <img src="https://img.shields.io/badge/GitHub-Repository-gray?logo=github" alt="GitHub: Kitsune-TTS"/> </a> </p>

Kitsune-TTS V1

A compact, non-autoregressive speech model built on VITS2-Slim, trained on synthetic data for expressive anime-style voices. Run it locally with PyTorch or use the ONNX export for CPU and web inference.

This repository contains the official model weights and matching configuration. The Python API, JavaScript client, training tools and examples live in the GitHub repository.

At a glance

📊 Benchmarks & Performance

Measured synthesizing 9.21 seconds of audio (noise_scale=0, speaker Frieren, FP32 computation, zero quantization, identical weights):

💡 Highlights:

• PyTorch fast_cpu: Delivers a ~1.96× speedup over standard PyTorch CPU, outperforming ONNX Runtime while keeping 100% identical audio fidelity.
• GPU Inference: Generates ~10 seconds of speech in just 200 ms.
• Full reproduction scripts and raw data: BENCHMARK_RESULTS.md.

Meet the voices

Use the speaker key with the Python API, or the numeric ID with the JavaScript client.

Choose your format

Keep model_config.json beside the checkpoint or ONNX file. Always use weights and configuration from the same model release.

Quick start

Install eSpeak NG on your system for phonemization, then set up the repository:

git clone https://github.com/Heitorkk2/Kitsune-TTS.git
cd Kitsune-TTS
pip install -e ".[torch,onnx]" huggingface_hub scipy

PyTorch

Download the compact checkpoint and configuration:

hf download Heitorkk2/Kitsune-TTS-V1 latest_model_fp16.pth model_config.json --local-dir model

Generate your first WAV:

from scipy.io.wavfile import write
from kitsune.api import KitsuneSynthesizer

synth = KitsuneSynthesizer(
    checkpoint="model/latest_model_fp16.pth",
    device="cpu",
    fast_cpu=True,  # 🚀 ~2x faster vocoder on CPU
)
print(synth.list_speakers())

audio = synth.synthesize("Olá! Eu sou a Frieren.", speaker="frieren")
write("output.wav", synth.sample_rate, audio)

ONNX

Download the graph and configuration:

hf download Heitorkk2/Kitsune-TTS-V1 kitsune39M.onnx model_config.json --local-dir model

Run with ONNX Runtime:

from kitsune.api import KitsuneSynthesizer

synth = KitsuneSynthesizer(onnx_path="model/kitsune39M.onnx")
audio = synth.synthesize("Uma voz pequena, com muito a dizer.", speaker="emilia")

For browser/Node.js usage and export instructions, see the JavaScript client and ONNX guide.

Add your own voice

Configure your recordings and transcripts in the Colab notebook to create a separate checkpoint for one or more new speakers. The model architecture stays the same; each new speaker adds one embedding row.

Fine-tuning is experimental; defaults may evolve as we test more voices.

Usage notes

• V1 supports Brazilian Portuguese. Other languages are not supported by this release.
• Speech quality, pacing and pronunciation can vary with the input text.
• Runtime performance depends on your processor, backend, thread settings and text length.
• Use voice recordings and datasets you have permission to use.

Special thanks

Special thanks to Everteson and Nakamura for helping build the Kitsune-TTS model.

Credits & licensing

The Kitsune model weights, code and synthetic dataset are original work, licensed under GPL-3.0.

• Architecture: daniilrobnikov/vits2, MIT. Used as the architectural starting point via weight transplant from a VCTK-pretrained checkpoint. Text encoder, flow and posterior encoder layers were carried over; duration prediction, speaker embeddings and vocabulary embeddings were re-initialized from scratch to support the new model.
• Phonemization: eSpeak NG (GPL-3.0), through the phonemizer library.
• Initial transplant checkpoint: VCTK Corpus, CC BY 4.0.

Acknowledgments

• VITS2: base architecture inspiration.
• OmniVoice (k2-fsa): zero-shot voice cloning used to bootstrap the training corpus; no real recordings were used for that corpus.
• XTTS (Coqui): additional synthetic dataset generation.
• Kokoro TTS: inspiration for compact speech models.
• Piper TTS: ONNX export and CPU inference reference.

---

<div align="center">

ᓚ₍ ^. ̫ .^₎

Made with ❤️ by Heitorkk2, Nakamura & Everteson

</div>