LAION-Box-Emotional-v0.7-r64

LAION-Box Emotional v0.7 — rank-64 (expressive voice-acting TTS, fully-merged)

Rank-64 variant of the emotional fine-tune. Three fully-merged standalone LTX-2.3 3.3B audio-only DiT checkpoints — use directly, no LoRA loading. (Companion repos: rank-32 = laion/LAION-Box-Emotional-v0.7, rank-64/128 = -r64 / -r128.)

Lineage

Lightricks/LTX-2.3 (3.3B audio-only DiT) → DramaBox (Resemble AI, IC-LoRA fine-tune of LTX-2.3) → LAION run16 v0.7 LoRA (rank 256, merged) → emotion LoRA rank 64, α 64 (10 epochs, merged).

Files

Recommended: best1_step1150. Also inference.py, download_components.py.

Components needed for inference

• audio DiT — this repo's *Emotional*.safetensors (✅ included)
• VAE + vocoder — dramabox-audio-components.safetensors (✅ included)
• text/prompt encoder — unsloth/gemma-3-12b-it-bnb-4bit (Google Gemma 3 12B, ~7.4 GB) — download_components.py
• reference denoiser — nvidia/RE-USE (SEMamba) — download_components.py
• pipeline code — DramaBox / LTX-2.3 (ltx2 + src/) from ResembleAI/Dramabox

Google Gemma + NVIDIA RE-USE are fetched from their canonical repos (not re-hosted), under their own licenses.

Usage

import sys; sys.path.insert(0, "DramaBox/src")
from inference_server import TTSServer
tts = TTSServer(checkpoint="LAION-Box-Emotional-v0.7-r64_best1_step1150.safetensors",
                full_checkpoint="dramabox-audio-components.safetensors",
                gemma_root="<gemma dir>", device="cuda", dtype="bf16", bnb_4bit=True)
tts.generate_to_file(prompt="A woman, voice breaking: 'You finally came back.'",
                     output="out.wav", voice_ref="reference_voice.wav",
                     cfg_scale=2.5, stg_scale=1.5, denoise_ref=True, seed=42)

Put the emotional direction in the prompt (e.g. "furious, shouting", "tender and hushed"). cfg_scale ↑ = follows the emotional prompt harder.

Training

rank 64, α 64, lr 1e-4, 6× GPU, grad-accum 8, bf16, 10 epochs on the 5,596-sample high-emotion subset (top-10%/30% by EmoNet intensity via laion/Empathic-Insight-Voice-Plus), mixed with ~35 % freshly-streamed German (≈3,000 new, non-repeating Emolia German samples per emotional epoch, generated live by a Gemma producer) so German quality is retained. Best-3 by flow-matching loss. Emotional data: TTS-AGI/emotional-voice-acting-subset-v0.7 (private).

Credits & license

LTX-2.3 © Lightricks (LTX-2 Community License) · DramaBox © Resemble AI · Gemma © Google · RE-USE © NVIDIA. Emotional fine-tune by LAION. Research/eval use.