chatterbox-tts-apple-silicon-code

🎙️ Chatterbox-TTS Apple Silicon

High-quality voice cloning with native Apple Silicon MPS GPU acceleration!

This is an optimized version of ResembleAI's Chatterbox-TTS specifically adapted for Apple Silicon devices (M1/M2/M3/M4) with full MPS GPU support and intelligent text chunking for longer inputs.

✨ Key Features

🚀 Apple Silicon Optimization

• Native MPS GPU Support: 2-3x faster inference on Apple Silicon
• CUDA→MPS Device Mapping: Automatic tensor device conversion
• Memory Efficient: Optimized for Apple Silicon memory architecture
• Cross-Platform: Works on M1, M2, M3 chip families

🎯 Enhanced Functionality

• Smart Text Chunking: Automatically splits long text at sentence boundaries
• Voice Cloning: Upload reference audio to clone any voice (6+ seconds recommended)
• High-Quality Output: Maintains original Chatterbox-TTS audio quality
• Real-time Processing: Live progress tracking and chunk visualization

🎛️ Advanced Controls

• Exaggeration: Control speech expressiveness (0.25-2.0)
• Temperature: Adjust randomness and creativity (0.05-5.0)
• CFG/Pace: Fine-tune generation speed and quality (0.2-1.0)
• Chunk Size: Configurable text processing (100-400 characters)
• Seed Control: Reproducible outputs with custom seeds

🛠️ Technical Implementation

Core Adaptations for Apple Silicon

1. Device Mapping Strategy

# Automatic CUDA→MPS tensor mapping
def patched_torch_load(f, map_location=None, **kwargs):
    if map_location is None:
        map_location = 'cpu'  # Safe fallback
    return original_torch_load(f, map_location=map_location, **kwargs)

2. Intelligent Device Detection

if torch.backends.mps.is_available():
    DEVICE = "mps"  # Apple Silicon GPU
elif torch.cuda.is_available():
    DEVICE = "cuda"  # NVIDIA GPU
else:
    DEVICE = "cpu"   # CPU fallback

3. Safe Model Loading

# Load to CPU first, then move to target device
MODEL = ChatterboxTTS.from_pretrained("cpu")
if DEVICE != "cpu":
    MODEL.t3 = MODEL.t3.to(DEVICE)
    MODEL.s3gen = MODEL.s3gen.to(DEVICE)
    MODEL.ve = MODEL.ve.to(DEVICE)

Text Chunking Algorithm

• Sentence Boundary Detection: Splits at .!? with context preservation
• Fallback Splitting: Handles long sentences via comma and space splitting
• Silence Insertion: Adds 0.3s gaps between chunks for natural flow
• Batch Processing: Generates individual chunks then concatenates

🚀 app.py Enhancements Summary

Our enhanced app.py includes:

• 🍎 Apple Silicon Compatibility - Optimized for M1/M2/M3/M4 Macs
• 📝 Smart Text Chunking with sentence boundary detection
• 🎨 Professional Gradio UI with progress tracking
• 🔧 Advanced Controls for exaggeration, temperature, CFG/pace
• 🛡️ Error Handling with graceful CPU fallbacks
• ⚡ Performance Optimizations and memory management

💡 Apple Silicon Note

While your Mac has MPS GPU capability, chatterbox-tts currently has compatibility issues with MPS tensors. This app automatically detects Apple Silicon and uses CPU mode for maximum stability and compatibility.

🎵 Usage Examples

Basic Text-to-Speech

1. Enter your text in the input field
2. Click "🎵 Generate Speech"
3. Listen to the generated audio

Voice Cloning

1. Upload a reference audio file (6+ seconds recommended)
2. Enter the text you want in that voice
3. Adjust exaggeration and other parameters
4. Generate your custom voice output

Long Text Processing

• The system automatically chunks text longer than 250 characters
• Each chunk is processed separately then combined
• Progress tracking shows chunk-by-chunk generation

📊 Performance Metrics

🔧 System Requirements

Minimum Requirements

• macOS: 12.0+ (Monterey)
• Python: 3.9-3.11
• RAM: 8GB
• Storage: 5GB for models

Recommended Setup

• macOS: 13.0+ (Ventura)
• Python: 3.11
• RAM: 16GB
• Apple Silicon: M1/M2/M3/M4 chip
• Storage: 10GB free space

🚀 Local Installation

Quick Start

# Clone this repository
git clone <your-repo-url>
cd chatterbox-apple-silicon

# Create virtual environment
python3.11 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Run the app
python app.py

Dependencies

torch>=2.0.0          # MPS support
torchaudio>=2.0.0     # Audio processing
chatterbox-tts        # Core TTS model
gradio>=4.0.0         # Web interface
numpy>=1.21.0         # Numerical ops
librosa>=0.9.0        # Audio analysis
scipy>=1.9.0          # Signal processing

🔍 Troubleshooting

Common Issues

Model Loading Errors

• Ensure internet connection for initial model download
• Check that MPS is available: torch.backends.mps.is_available()

Memory Issues

• Reduce chunk size in Advanced Options
• Close other applications to free RAM
• Use CPU fallback if needed

Audio Problems

• Install ffmpeg: brew install ffmpeg
• Check audio file format (WAV recommended)
• Ensure reference audio is 6+ seconds

Debug Commands

# Check MPS availability
python -c "import torch; print(f'MPS: {torch.backends.mps.is_available()}')"

# Monitor GPU usage
sudo powermetrics --samplers gpu_power -n 1

# Check dependencies
pip list | grep -E "(torch|gradio|chatterbox)"

📈 Comparison with Original

🤝 Contributing

We welcome contributions! Areas for improvement:

• MLX Integration: Native Apple framework support
• Batch Processing: Multiple inputs simultaneously
• Voice Presets: Pre-configured voice library
• API Endpoints: REST API for programmatic access

📄 License

MIT License - feel free to use, modify, and distribute!

🙏 Acknowledgments

• ResembleAI: Original Chatterbox-TTS implementation
• Apple: MPS framework for Apple Silicon optimization
• Gradio Team: Excellent web interface framework
• PyTorch: MPS backend development

📚 Technical Documentation

For detailed implementation notes, see:

• APPLE_SILICON_ADAPTATION_SUMMARY.md - Complete technical guide
• MLX_vs_PyTorch_Analysis.md - Performance comparisons
• SETUP_GUIDE.md - Detailed installation instructions

---

🎙️ Experience the future of voice synthesis with native Apple Silicon acceleration!

This Space demonstrates how modern AI models can be optimized for Apple's custom silicon, delivering superior performance while maintaining full compatibility and ease of use.