kanade-25hz-clean

Kanade: A Simple Disentangled Tokenizer for Spoken Language Modeling

Paper   Code   Demo  

Kanade is a single-layer disentangled speech tokenizer that extracts compact tokens suitable for both generative and discriminative modeling.

Updates

• 2026-01-09: Released kanade-25hz-clean model trained on LibriTTS-R with HiFT vocoder for better audio quality. LibriTTS-R is a restored version of LibriTTS removing noise, so the model trained on it can produce cleaner synthesis. Because of that, however, this version can no longer faithfully reflect the recording environment such as background noise and microphone characteristics. Also, the vocoder is changed to the HiFT model used in CosyVoice 2 for better quality. The content encoder part remains the same as the previous kanade-25hz model. We made tiny code change to support different vocoders during inference (specifically load_vocoder). Please refer to the updated usage section below.

Models

Installation

For simple inference, you can install Kanade tokenizer to your virtual environment:

# In your own project's virtual environment
uv add git+https://github.com/frothywater/kanade-tokenizer
# or using pip
pip install git+https://github.com/frothywater/kanade-tokenizer

[!IMPORTANT] We use FlashAttention for efficient local window attention in our training. We recommend installing it following the instructions in their repository to get the best performance and the closest match to our setup. The model will fall back to regular PyTorch SDPA implementation if FlashAttention is not available. In this case, we cannot guarantee the same quality as reported in the paper.
If using uv, you can install FlashAttention like: uv pip install flash-attn --no-build-isolation. (Ensure ninja is installed in your system or the build will be very slow.)

Usage

Example code to load the model from HuggingFace Hub and run inference:

from kanade_tokenizer import KanadeModel, load_audio, load_vocoder, vocode

# Load Kanade model
model = KanadeModel.from_pretrained("frothywater/kanade-25hz-clean")
model = model.eval().cuda()

# Load vocoder
vocoder = load_vocoder(model.config.vocoder_name).cuda()

# Load audio (samples,)
audio = load_audio("path/to/audio.wav", sample_rate=model.config.sample_rate).cuda()

# Extract features
features = model.encode(waveform)

# Synthesize audio from extracted features
mel_spectrogram = model.decode(
    content_token_indices=features.content_token_indices, # (seq_len,)
    global_embedding=features.global_embedding, # (dim,)
) # (n_mels, T)

# Resynthesize waveform using vocoder
resynthesized_waveform = vocode(vocoder, mel_spectrogram.unsqueeze(0)) # (1, samples)

For details about voice conversion and how to train and fine-tune the model, please refer to our github repository.