streaming-vad-crnn

Streaming Neural VAD — Causal CRNN (47,873 params)

A small-footprint, causal, streaming-oriented Voice Activity Detector built for an on-device far-field target. Four causal depthwise-separable Conv1D residual blocks (kernel 5, dilations {1,2,4,8}) followed by a unidirectional GRU (hidden 64) and a per-frame linear head, over 64-bin log-mel features at a 10 ms frame rate.

• Parameters: 47,873 (~0.19 MB FP32)
• Look-ahead: 0 ms in the network; 12.5 ms from the centred 25 ms analysis window
• CPU end-to-end RTF: 0.0035 on 60 s inputs (single thread, Xeon Gold 6248)
• Framework: PyTorch 2.8

Intended use

Frame-level speech/non-speech decisions upstream of wake-word / ASR gating. Trained on clean read English (LibriSpeech-style) with additive MUSAN noise; it is a research prototype, not validated on real far-field, multi-speaker, or reverberant audio (see Limitations).

Results

Held-out, speaker-disjoint clean test partition (93 utterances, 4 speakers). All operating thresholds are selected on validation only, then frozen.

Continuous-score: CRNN AUROC 0.984 / AUPRC 0.997; Silero 0.982 / 0.996. A paired 10k-utterance bootstrap gives a 95% CI of [0.0009, 0.0035] for the clean F1 difference (CRNN − calibrated Silero).

Noise robustness (frame F1, held-out MUSAN, global-RMS SNR):

Full analysis, curves, and caveats are in the accompanying technical report.

Usage

from inference import VAD
vad = VAD("vad_crnn_export")            # dir with model.safetensors + config.json
probs, mask, times = vad("clip.wav")    # frame probs, binary mask, frame centres (s)

vad_model.py must contain your real architecture — see its docstring.

Files

• vad_crnn_export/model.safetensors, vad_crnn_export/config.json — weights + config
• inference.py — standalone loader / predictor
• vad_model.py — architecture factory (you paste your crnn.py here)
• export_checkpoint.py — regenerates the export from the training checkpoint

Limitations

Clean read English only; additive-noise robustness under a global-RMS SNR definition; reverberation untested in the released model; four test speakers; single training seed; server-CPU (not target-device) latency; no stateful chunked-streaming benchmark. Do not treat in-domain F1 as deployed performance.

Training data & attribution

Provided challenge corpus (LibriSpeech-derived) + MUSAN music/noise for augmentation. The corpus and any derived mixtures are not redistributed here. LibriSpeech (Panayotov et al., 2015), MUSAN (Snyder et al., 2015).