vocalburst-classifier-single

VocalBurst Classifier — single-burst

A sharper classifier specialised for clips that contain exactly one isolated vocal burst (no speech). Same taxonomy (82 classes + no_burst). Softmax (single most-likely type). Use it as the "name the burst" stage after a detector has isolated a segment.

⚠️ Taxonomy patch (label mapping only, weights unchanged): the three classes originally trained at indices 79 Hand Scratching Head, 80 Hand Slaps and 81 Slap Face are now labelled No burst because of excessive false positives in production. All 83 logits and indices are intact. See Taxonomy patch before using raw logits.

Architecture

laion/voiceclap-commercial (768-d, frozen) → MLP 1024-wide × 3 deep → 83-way softmax. ~3M params.

Training

Trained ONLY on clean single-burst clips: Gemini-confirmed DACVAE positives (82 classes, exactly one label, no speech) + pure Emilia no_burst clips (~15.9k). Cleaner labels, no speech confusion → sharper. 50 epochs, AdamW, cross-entropy, best-mAP checkpoint.

v2 (current): retrained with 9,000 multilingual burst-free negatives (FLEURS: Chinese, Hindi, Bengali, Arabic, Persian, Urdu, Tamil, Telugu, Vietnamese, Thai, Indonesian, Japanese, Korean, Swahili, Yoruba, Zulu, Turkish, Russian) — fixes hallucinated bursts on non-European speech.

Results (held-out pure single-burst clips)

How it works

Two stages: a frozen laion/voiceclap-commercial audio encoder turns a clip into a 768-d embedding (encode_waveform, auto-downloaded — the repo needs no extra setup), then this small trained MLP head maps it to 83 outputs = 82 VocalBurst classes + no_burst (taxonomy: LAION-AI/voice-taxonomies · vocalburst). A no-burst gate: if P(no_burst) ≥ 0.5 the clip is declared burst-free (no false alarm); otherwise the top classes are returned. Clips are truncated to the first 30 s (the encoder's window).

Usage

from inference import VocalBurstClassifier
clf = VocalBurstClassifier("laion/vocalburst-classifier-single")      # HF repo id, or a local checkout dir
print(clf.predict("clip.wav"))        # -> {no_burst, p_no_burst, top1, predictions:[(class,prob)], group}

Taxonomy patch: indices 79 / 80 / 81 are now labelled No burst

This section supersedes the earlier "Known caveat: Slap Face false positives" note. That note recommended skipping Slap Face as top-1 and falling through to the runner-up. The problem turned out to be broader than one class and is now fixed in the label mapping itself, so the manual skip rule is no longer needed — do not apply it on top of this.

What changed

The model was originally trained with three fine classes in the hand_and_body_sounds family:

In large-scale production use — re-classifying ~1.5 M CrisperWhisper-detected burst timestamps from in-the-wild expressive speech — all three fired as top-1 far more often than is plausible, and manual spot-checks showed the great majority were false positives: the true event was usually a laugh, gasp, sigh, grunt, or simply no burst at all. Slap Face was the worst offender, but Hand Slaps and Hand Scratching Head behaved the same way. These three are percussive, broadband, very short events, and the frozen embedder appears to map a lot of unrelated transient energy (mouth clicks, mic bumps, plosives, clipping) into that corner of the space.

Accordingly, in classes.json all three labels have been replaced with "No burst", and class_to_group.json maps "No burst" to the group no_burst.

What did not change

The weights were not retrained and not altered. model.pt is bit-identical to the previous revision. The head still emits 83 logits and every index keeps its original position, so existing checkpoints, cached embeddings, cached logits and stored predicted_index values all stay valid. This is purely a relabelling of the output mapping.

What this means if you use the raw logits

• argmax still returns an integer in [0, 82]. Indices 79, 80 and 81 now decode to the string "No burst" instead of the three hand-sound names.
• "No burst" therefore appears three times in classes.json. Do not use classes.index("No burst") to recover an index — it will always give 79. Always keep the integer index as the identity and use the list only for display.
• Index 82 is still the model's own dedicated no_burst class, spelled no_burst (lowercase, underscore) to keep the pre-existing gate code working: P(no_burst) ≥ 0.5 still refers to index 82 alone.
• The full set of "this is not a burst" outcomes is therefore {79, 80, 81, 82}. If you want a single boolean, test membership in that set (or test class_to_group[label] == "no_burst"), not equality against one index.
• If you need the original three-way hand-sound distinction back (e.g. your domain really does contain slaps and you have evaluated it), the information is still in the logits — just restore the names at indices 79/80/81 in your own copy of classes.json.
• The coarse family hand_and_body_sounds now contains only Finger Snaps (index 78).

probs = model(emb).softmax(-1)
idx   = int(probs.argmax(-1))          # keep the INDEX as the identity
label = classes[idx]                   # 79/80/81 -> "No burst"
is_burst = idx not in (79, 80, 81, 82)

This is an observation from downstream use on in-the-wild speech, not a formal error analysis on the model's own held-out split — the reported metrics above were computed before this relabelling and still refer to the original 83-way taxonomy.

Files

model.pt (MLP weights, unchanged) · config.json (arch) · classes.json (83 labels, index order; 79/80/81 relabelled No burst) · class_to_group.json (fine→coarse families) · inference.py · example.py · requirements.txt.

Interactive demos (audio + predictions)

• Multi-label predictions vs ground truth: https://projects.laion.ai/procedural-voice-captions/vocalburst-predictions/
• Single-burst classifier: https://projects.laion.ai/procedural-voice-captions/vocalburst-single/
• Two-stage detect→classify combo: https://projects.laion.ai/procedural-voice-captions/vocalburst-combo/
• On MOSS character voices: https://projects.laion.ai/procedural-voice-captions/vocalburst-character/

Training data + embeddings: laion/vocalburst-classification. Embedder: laion/voiceclap-commercial. License: CC-BY-4.0.