teensy-vad-2

teensy-vad-2 — distilled from Silero VAD

Same tiny architecture as teensy-vad-1 (20,449 params, 87 KB, pure numpy), but trained on Silero VAD teacher labels instead of construction labels — knowledge distillation from a ~2M-parameter teacher into a 20k-parameter student.

Why: construction labels know where the utterance was placed, not where speech actually starts and stops. Silero relabelled every frame of the training mixtures (soft probabilities, interpolated to the 10 ms grid); the teacher disagreed with construction labels on 11.6 % of frames — 9.2 % were pauses/breathy tails mislabelled "speech". The student learned where speech actually is:

Quick Start

from teensyvad import OfflineVAD

# Standalone VAD — auto-downloads from the Hub (pip install huggingface_hub)
model = OfflineVAD("Teensy/teensy-vad-2")
result = model.segments("long_audio.wav")
# Returns speech segments: [[start_ms, end_ms], [start_ms, end_ms], ...]
print(result)                    # e.g. [[90, 5150]] — the fsmn-vad convention

Variant selection: OfflineVAD("Teensy/teensy-vad-2", model_file="teensy-v2-80k.npz") (see the variant table below). Local use without the hub: pass a .npz path. teensyvad is pure numpy (single package dir in the project repo) — the only dependency.

Use as Part of an ASR Pipeline

from teensyvad import OfflineVAD
from teensyvad.audio import write_wav

vad = OfflineVAD("Teensy/teensy-vad-2")
segments = vad.segments("meeting_2hours.wav")     # [[start_ms, end_ms], ...]

for i, (start_ms, end_ms) in enumerate(segments):
    seg = vad.slice("meeting_2hours.wav", start_ms, end_ms)  # float32 @ 8 kHz
    write_wav(f"speech_{i}.wav", seg, 8000)
    # text = my_asr.transcribe(f"speech_{i}.wav")  # <- your ASR here

Streaming / telephony (Asterisk AudioSocket): from teensyvad import StreamingVAD — feed 20 ms PCM16LE frames, get speech_start / speech_end events, vad.speech_seconds for talk time; a working server ships as scripts/audiosocket_server.py in the project repo.

Architecture

Identical to v1 (see its card for the exact, reimplementable feature spec): 400→48→24→1 MLP over 10 frames of 20 log-mel+Δ at 8 kHz.

What's in this repo

Training

• Same mixtures as v1 (LibriSpeech dev-clean + ESC-50, 0–20 dB SNR, µ-law augmentation), labels from Silero VAD run at 8 kHz
• Two students trained: hard 0/1 labels (this model) and soft probabilities (slightly higher AUC 0.929 vs 0.922 on val; the hard variant produced tighter event boundaries and is the default)
• Event thresholds calibrated at event level on validation

Measured quality

Teacher (Silero) on the same real sets for scale: TEN F1 0.937 / AUC 0.863; AMI F1 0.714 / AUC 0.772 — the student already out-ranks its teacher on AMI (real rooms) while the teacher stays better calibrated close-mic.

Quantization

PTQ is essentially free on this model (ΔAUC 0.0000, 99.7 % decision agreement); QAT adds +0.9 F1 points over PTQ at identical size. In pure numpy int8 is a size play (~3× smaller) not a speed play — for speed use the ONNX int8 file (real int8 kernels, 0.067 µs/frame batched).

Limitations (honest)

• A student inherits its teacher's biases — Silero's conservatism on sung/tone-like audio transfers.
• English read speech + environmental noise; no babble or room tone (see teensy-vad-3).
• Operating thresholds are domain-specific: v2's real-set numbers above used AMI-calibrated thresholds (0.10); the stored default suits synthetic-event use.

Models in this repository (named variants)

All variants share the architecture and distillation recipe of this card — float variants differ only in hidden-layer size (88/48, 164/88, 200/96); the QAT variant is the 20k net fine-tuned under int8 simulation:

ONNX exports (float32 + dynamic int8) are provided for the 20k model.

Accuracy & speed of every variant

(Measured on the shared protocol — TEN VAD public set + AMI SDM meetings, human labels, AMI-dev-calibrated operating points, AUC on raw probabilities; see BENCHMARKS.md. Accuracy is flat from 20k→100k on the 1M-frame training set — the distilled 20k is the family's sweet spot; scale data instead, see teensy-vad-3.)

Comparison vs Energy / WebRTC / Silero

Same protocol for every system, human-labelled real audio, operating points calibrated on held-out AMI dev meetings (full appendix with charts and per-size results: teensy-vad-3/BENCHMARKS.md):

* like-for-like with FlashVAD's published TEN numbers (also threshold-tuned on that set): FlashVAD F1 0.889 / AUC 0.882 — teensy-v2 matches the F1 with 2.3× fewer parameters.

Capacity note: this family was also trained at 40k/80k/100k params (teensy-v2-{40k,80k,100k}.npz) — like v1, accuracy is flat across sizes on the 1M-frame training set. The distilled 20k model is the family's sweet spot; scale the data instead (see teensy-vad-3).

License & data

Code: MIT. Weights: CC BY-NC-SA 4.0 (ESC-50 training noise is CC BY-NC-SA 3.0; speech LibriSpeech CC BY 4.0; teacher Silero VAD is MIT). Not for commercial deployment without retraining on permissive noise data.