higgs-v2-tokenizer

Higgs Audio V2 Tokenizer

This is the tokenizer for the Higgs Audio V2 text-to-speech model.

Description

The Higgs Audio V2 Tokenizer handles text preprocessing and tokenization for the Higgs Audio V2 TTS model.

Model Details

• Model Type: Text Tokenizer
• Framework: HuggingFace Transformers
• Purpose: Text preprocessing for TTS

Usage

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Toowired/higgs-v2-tokenizer")

# Tokenize text for TTS
text = "Hello, this is a test."
tokens = tokenizer(text, return_tensors="pt")

Features

• Optimized for English text
• Handles special characters and punctuation
• Compatible with Higgs Audio V2 LLM
• Fast tokenization for real-time applications

Intended Use

• Text preprocessing for text-to-speech
• Tokenization for Higgs Audio V2 model
• Research in TTS preprocessing

Technical Details

• Vocabulary size optimized for speech synthesis
• Special tokens for audio markers
• Efficient encoding/decoding

License

Please check the original model repository for license information.

Contact

For questions about this tokenizer, please contact the repository owner.