voiceclap-commercial-genuineness

VoiceCLAP-commercial - Speech Genuineness Predictor

Predicts a 0-6 genuineness score for a speech clip: how much it sounds like a real, lived-in spoken moment (6) versus a rehearsed or synthetic read (0). Genuineness is about believability and lived-in delivery - natural timing, breath, micro-imperfections, and emotional grounding - not about audio fidelity.

This repository is fully standalone. It bundles the original frozen laion/voiceclap-commercial audio embedder under voiceclap_commercial/ plus a small trained MLP head, so inference needs no network access. This is the lightweight (~450 MB) sibling of the large-v2 genuineness model.

What it predicts

A single continuous score in [0, 6]:

• 0-2 - reads as rehearsed, flat, or synthetic
• 3-4 - plausibly natural, some life to it
• 5-6 - sounds like a genuine, unscripted, lived-in moment

How it was trained

• ~10k speech clips drawn from a mix of various TTS systems and the Emolia expressive-speech collection, spanning a wide range of naturalness.
• Each clip labelled 0-6 for genuineness by Gemini-3.1-Pro.
• Every clip is encoded once with the frozen VoiceCLAP-commercial embedder (768-d). A small MLP head - Linear(768,50) -> GELU -> Dropout(0.2) -> Linear(50,1) - is trained on top with Huber loss (delta 1.5, Adam, standardized inputs). The VoiceCLAP backbone is never fine-tuned.
• Two heads ship: full (default, trained on the full label distribution) and balanced (retrained on a class-balanced subset - flatter per-bucket error, similar overall MAE).

Validation metrics (this model)

Held-out val set = 140 clips (20 per genuineness value 0-6, stratified, seed 1234). Full head:

See val_predictions.html for every val clip with an audio player, ground-truth vs predicted score (sorted by prediction), a pred-vs-GT scatter, and per-bucket errors.

Usage

from genuineness_scorer import GenuinenessScorer

scorer = GenuinenessScorer(pkg_dir=".", device="cuda")   # loads bundled VoiceCLAP + head locally
print(scorer.score("clip.wav"))                          # -> float in [0, 6]
print(scorer.score_batch(["a.wav", "b.wav"]))            # -> [float, float]

# class-balanced head instead of the default full-data head:
scorer_bal = GenuinenessScorer(pkg_dir=".", model="balanced", device="cuda")

Or from the shell:

pip install -r requirements.txt
python example.py clip.wav          # prints genuineness (0-6)

The embedder loads from the local voiceclap_commercial/ folder with trust_remote_code=True; set HF_HUB_OFFLINE=1 to guarantee no network access at inference time.

Files

• voiceclap_commercial/ - bundled frozen VoiceCLAP-commercial embedder (~450 MB)
• genuineness_head.pt - full-data MLP head (default)
• genuineness_head_balanced.pt - class-balanced MLP head
• genuineness_scorer.py - GenuinenessScorer inference class
• example.py, requirements.txt, val_predictions.html

Limitations

• Label skew: training labels lean low (most clips fall in 0-2), so the model is most reliable there and noisier on rare high-genuineness clips (per-bucket MAE grows toward g6).
• The commercial backbone is smaller and trades some accuracy for size; the large-v2 model scores meaningfully better (MAE 0.81 / r 0.83).
• Genuineness is a subjective, model-labelled construct; treat scores as a soft ranking signal, not ground truth. Out-of-domain audio (heavy noise, music, non-speech) is out of scope.

License

CC-BY-4.0. The bundled VoiceCLAP-commercial weights retain their original license from laion/voiceclap-commercial.