teensy-vad-3

teensy-vad-3 — scaled data, real-world hardened

The best model of the teensyvad family: the same 20,449-parameter architecture as v1 and v2, trained on 10× more and much harder data — and evaluated on human-labelled real recordings.

Headline: at its AMI-calibrated operating point this 87 KB student scores AMI F1 0.886 vs Silero's 0.714 (Silero's stock threshold misses 44 % of speech in real rooms), and beats WebRTC VAD on every metric — at 1/20th the size of the Silero model. On ranking quality (AUC, computed on raw probabilities for every system) Silero remains ahead (see the comparison below) — the honest split is: Silero ranks best; teensy-v3 operates best in rooms, per KB and per µs.

(FlashVAD v0.1, for reference, publishes F1 0.889 / AUC 0.882 on the TEN set — threshold-tuned on that set, as is our 0.894 best-threshold number: parity within caveats, at 2.3× fewer parameters.)

Training (what changed vs v2)

• Speech: 8,000 utterances from LibriSpeech train-clean-100 (~30 h of mixtures, 10.7M teacher-labelled frames — 10× v2)
• Noise, three families:
  • ESC-50 environmental (as before, fold-disjoint)
  • synthetic 7-talker babble — summed disjoint LibriSpeech speakers; the classic hard case for VAD (background conversations)
  • real AMI room ambience — non-speech stretches of distant-mic meeting audio (chairs, keyboards, HVAC), taken only from the 3 meetings reserved for calibration (the 8 evaluation meetings were never seen)
• SNR −5 … 20 dB (down from 0 dB floor)
• Labels: Silero teacher (as in v2), µ-law augmentation kept (G.711 round-trip shifts real-set numbers by < 0.5 %)
• Lazy context-window training (LazyWindows — the full design matrix would be 17 GB; windows materialise per batch)

Quick Start

from teensyvad import OfflineVAD

# Standalone VAD — auto-downloads from the Hub (pip install huggingface_hub)
model = OfflineVAD("Teensy/teensy-vad-3")
result = model.segments("long_audio.wav")
# Returns speech segments: [[start_ms, end_ms], [start_ms, end_ms], ...]
print(result)                    # e.g. [[90, 5150]] — the fsmn-vad convention

Variant selection: OfflineVAD("Teensy/teensy-vad-3", model_file="teensy-v3-80k.npz") (see the variant table below). Local use without the hub: pass a .npz path. teensyvad is pure numpy (single package dir in the project repo) — the only dependency.

Use as Part of an ASR Pipeline

from teensyvad import OfflineVAD
from teensyvad.audio import write_wav

vad = OfflineVAD("Teensy/teensy-vad-3")
segments = vad.segments("meeting_2hours.wav")     # [[start_ms, end_ms], ...]

for i, (start_ms, end_ms) in enumerate(segments):
    seg = vad.slice("meeting_2hours.wav", start_ms, end_ms)  # float32 @ 8 kHz
    write_wav(f"speech_{i}.wav", seg, 8000)
    # text = my_asr.transcribe(f"speech_{i}.wav")  # <- your ASR here

Streaming / telephony (Asterisk AudioSocket): from teensyvad import StreamingVAD — feed 20 ms PCM16LE frames, get speech_start / speech_end events, vad.speech_seconds for talk time; a working server ships as scripts/audiosocket_server.py in the project repo.

What's in this repo

Architecture and the exact reimplementable feature spec are identical to v1's card: 8 kHz, 25/10 ms framing, 20 log-mel + Δ with per-frame band-mean subtraction, 10-frame context, 400→48→24→1 MLP.

Domain threshold profiles (important)

Rankings transfer across domains; operating points do not. Best threshold measured: ~0.45 close-mic/telephony, 0.10 distant-room, 0.85 synthetic-events. The .npz metadata therefore ships profiles:

{"profiles": {
   "close_mic":    {"thr_hi": 0.45, "thr_lo": 0.27},   // default (telephony)
   "distant_room": {"thr_hi": 0.10, "thr_lo": 0.06}}}  // AMI-calibrated

distant_room was calibrated on 3 held-out AMI meetings; the 8 evaluation meetings were never used for any tuning.

Limitations (honest)

• At distant-room operating points everyone's false-alarm rate is high on overlapped meeting speech (labels mark foreground speech only).
• English speech; no music in training (a known gap — music reads as "activity").
• 100 ms context is short: unvoiced fricatives in noise remain the hardest frames; a GRU/TCN would help (deliberately out of scope — this family stays a readable MLP).
• µ-law/PSTN robustness verified; packet-loss concealment not modelled.

Models in this repository (named variants)

All variants share the 10.7M-frame scaled training recipe of this card — float variants differ only in hidden-layer size (88/48, 164/88, 200/96); the QAT variant is the 20k net fine-tuned under int8 simulation:

ONNX exports (float32 + dynamic int8) are provided for the 20k model.

Accuracy & speed of every variant

Unlike the v1/v2 families (flat 20k→100k on 1M frames), this family's 10.7M-frame training set lets capacity pay: TEN AUC rises 0.873 (20k) → 0.877 (80k) before saturating at 100k. Full cross-family story in BENCHMARKS.md (capacity.png).

Comparison vs Energy / WebRTC / Silero — and capacity scaling

Full protocol, charts and the complete 14-model table live in BENCHMARKS.md (same protocol for every system, human-labelled audio, AMI-dev-calibrated operating points, AUC on raw probabilities). Summary:

* tuned on that set — like-for-like with FlashVAD's published F1 0.889 / AUC 0.882.

This family was trained at 20k/40k/80k/100k params on the same 10.7M frames (teensy-v3-{40k,80k,100k}.npz): capacity pays here up to a sweet spot at ~80k (TEN AUC 0.873 → 0.877) then saturates — while the v1/v2 families (1M frames) are flat from 20k. Capacity scales only with data. See capacity.png in BENCHMARKS.md.

License & data

Code: MIT. Weights: CC BY-NC-SA 4.0 (ESC-50 CC BY-NC-SA 3.0 noise in training; LibriSpeech CC BY 4.0; AMI CC BY 4.0; Silero teacher MIT). Not for commercial deployment without replacing non-commercial training data.