tts-jarvis

TTS Jarvis V4 LoRA

A VoxCPM2 adapter for English assistant speech with a British male conversational voice target. The selected listening recipe combines this adapter with a fixed voice reference and the optional Forward post-processing preset.

Six inference steps are the selected default after a controlled 6/12/24-step comparison. Higher steps increased synthesis latency without establishing a clear audible improvement. The V4 adapter and reference recipe are unchanged.

Preview

Short response, the same line used in the usage example below:

<audio controls src="https://huggingface.co/FuturePresentLabs/tts-jarvis/resolve/main/samples/preview-short.wav"></audio>

"The workshop is ready."

Greeting:

<audio controls src="https://huggingface.co/FuturePresentLabs/tts-jarvis/resolve/main/samples/preview-greeting.wav"></audio>

"Good morning. What would you like to work on today?"

Multi-sentence assistant dialogue:

<audio controls src="https://huggingface.co/FuturePresentLabs/tts-jarvis/resolve/main/samples/preview-dialogue.wav"></audio>

"Good evening. The workshop is ready, and the overnight builds finished without errors. Three deployments are waiting on your approval when you have a moment."

Technical content:

<audio controls src="https://huggingface.co/FuturePresentLabs/tts-jarvis/resolve/main/samples/preview-technical.wav"></audio>

"The build completed in four minutes and twelve seconds, with two warnings in the audio pipeline."

Numbers and identifiers — one of the harder cases, included deliberately:

<audio controls src="https://huggingface.co/FuturePresentLabs/tts-jarvis/resolve/main/samples/preview-numbers.wav"></audio>

"Order 4417 ships Tuesday. That's 3 units at 89 dollars each, totalling 267 dollars."

Long-form, where artifacts are most likely to appear:

<audio controls src="https://huggingface.co/FuturePresentLabs/tts-jarvis/resolve/main/samples/preview-longform.wav"></audio>

"There's a particular satisfaction in a system that explains itself. When something fails at three in the morning, the difference between a good design and a clever one is whether the error message tells you where to look. Most of the work is making the obvious thing possible."

The last two are not cherry-picked. Numbers, names and long passages are the cases listed under Evaluation And Limitations below, and they are included so the weaknesses are audible rather than described.

All clips use this adapter over the openbmb/VoxCPM2 base with the settings in inference_config.json: six inference steps, a fixed voice reference, and the Forward post-processing preset. They are buffered output at 24 kHz, not the raw streaming path.

The reference recording matters as much as the adapter. These were rendered with the reference the adapter was selected against, which is not distributed here; synthesising with a different reference will not sound like them. That is a property of the recipe, not a defect.

What It Does

• Generates short responses and multi-sentence assistant dialogue.
• Supports reference-conditioned synthesis through VoxCPM2's reference-audio input.
• Provides an adapter-only checkpoint; the VoxCPM2 base weights are loaded separately.
• Includes the selected generation settings and an optional Forward rendering script.

The preferred voice depends on the reference recording as well as the adapter. Using a different recording changes the result. The enrollment recording is not bundled with the adapter. Supply reference audio you are authorized to use.

Usage

Install a VoxCPM2-compatible runtime. The tested upstream runtime revision is f772e498a45fbb5fb8e13fbf9b9c48be9fe33e69; local MPS/training patches were used for the original experiments. See inference_config.json for runtime provenance.

import json
from pathlib import Path
import soundfile as sf
from huggingface_hub import snapshot_download
from voxcpm.core import VoxCPM
from voxcpm.model.voxcpm2 import LoRAConfig

release = Path(snapshot_download(
    "FuturePresentLabs/tts-jarvis", revision="v4-interface-2026-09-06"
))
settings = json.loads((release / "inference_config.json").read_text())
base = snapshot_download(settings["base_model"], revision=settings["base_revision"])
adapter = release / "adapter"
config = json.loads((adapter / "lora_config.json").read_text())["lora_config"]
model = VoxCPM.from_pretrained(
    hf_model_id=base, load_denoiser=False, optimize=False,
    lora_config=LoRAConfig(**config), lora_weights_path=str(adapter),
)
audio = model.generate(
    text="The workshop is ready.", reference_wav_path="reference.wav",
    cfg_value=2.4, inference_timesteps=6, seed=42,
    normalize=True, denoise=False, max_len=600,
)
sf.write("output_raw.wav", audio, model.tts_model.sample_rate)

infer.py also renders the optional Forward preset using FFmpeg with rubberband and aexciter support. Forward changes pitch, EQ, dynamics and level; it is not the raw model output. The example requires an installed compatible runtime and your own local reference.wav, not a Transformers pipeline.

Technical Details

V4 continues the V3 step-20 adapter for 32 additional optimizer updates at learning rate 3e-5, with a fresh optimizer. LoRA rank/alpha are 32; LM and DiT attention projections are adapted, projection modules are not. Training used 29 targets and five held-out targets with fixed-reference conditioning.

step_0000031 is zero-indexed and contains those 32 additional updates. The selected inference reference differs from the reference used during V4 training. The recipe has not been retrained around that new reference.

Evaluation And Limitations

The selected recipe was exercised on 12 English prompts across three seeds, including short replies, names, numbers and longer passages. This is a small internal diagnostic test, not a public benchmark or a claim of general superiority.

• Longer synthesis can develop unwanted noise or harshness despite strong transcript and speaker-similarity scores.
• Short words may be omitted or pronounced ambiguously; names need listening checks.
• Text normalization can misread standalone No. as the abbreviation for "number". Use deliberately verbalized text when diagnosing pronunciation.
• Level matching improves between-clip loudness but does not remove model artifacts.
• Sentence-based generation reduced one measured spectral trend, but changes cadence and is not established as a general solution.
• Multilingual performance and production latency have not been validated here.

Do not use the model to misrepresent a speaker's identity or endorsement.

License

Adapter distribution retains the project's Apache-2.0 license metadata and included LICENSE. The VoxCPM2 base model also declares Apache-2.0. This does not grant rights to third-party recordings, voices, likenesses, names, or endorsements.