Downloads · 30 days
0
DISENT-KWS
tripathiji1312/DISENT-KWS
DISENT-KWS is a machine learning model from tripathiji1312. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for pytorch. The card lists the license as mit.
DISENT-KWS is a lightweight, real-time dual-gate Keyword Spotting (KWS) and Speaker Verification system. Designed for resource-constrained edge devices, it allows users to register custom wake words and speaker profil…
Access
Public
Updated Jun 21, 2026
Repo size
7.9 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt7.3 MB · 92%
At a glance
- Library
- pytorch
- License
- mit
- Access
- Public
- Created
- Jun 18, 2026
- Updated
- Jun 21, 2026
- SHA
- dd488769
Model card
From the Hugging Face model README
DISENT-KWS: Speech Disentanglement for Robust Custom Word Detection
Overview
DISENT-KWS is a lightweight, real-time dual-gate Keyword Spotting (KWS) and Speaker Verification system. Designed for resource-constrained edge devices, it allows users to register custom wake words and speaker profiles with just 5 audio samples.
The model uses a Disentangled Dual-Head Architecture built on a shared BC-ResNet-2 backbone. An adversarial Gradient Reversal Layer (GRL) and Contrastive Log-ratio Upper Bound (CLUB) mutual information minimizer ensure that phonetic (keyword) and speaker identity representations remain strictly orthogonal. This prevents the system from triggering on the wake word if spoken by an unauthorized user.
Architecture Overview
The system uses a dual-head disentangled architecture built on a shared BC-ResNet-2 encoder (total 1.806M parameters):
- Shared Encoder (BC-ResNet-2) — Broadcasted residual network, 33.8K params
- Temporal Block (Mamba SSM / Dilated Conv1D) — O(T) temporal context modeling, 10.3K params
- Phonetic Head (Causal Conformer) — Extracts keyword-discriminative embeddings zₚₕₙ ∈ ℝ¹⁹², 1,673K params
- Speaker Head (ECAPA-TDNN Lite) — Extracts speaker-discriminative embeddings zₛₚₖ ∈ ℝ¹⁹², 88.8K params
- Disentanglement Module (GRL + CLUB) — Adversarial gradient reversal + mutual information minimization forces zₚₕₙ ⟂ zₛₚₖ
- Dual-Gate Scorer — Weighted cosine similarity (
w_kw=0.30,w_spk=0.65) with EMA smoothing and DET-calibrated threshold (τ=0.2222)
Datasets Used
| Dataset | Usage | Samples | License |
|---|---|---|---|
| Google Speech Commands v2 | Keyword spotting pre-training | 105K utterances | CC BY 4.0 |
| VoxCeleb1 | Speaker verification training | 153K utterances (1,251 speakers) | CC BY 4.0 |
| LibriPhrase | Hard-negative triplet pairs (up to 3K triplets generated from metadata) | Apache 2.0 | |
| MUSAN | Noise augmentation | 109 hrs | CC BY 4.0 |
Final Performance Benchmarks
Evaluated on Google Speech Commands v2 test set (11,005 samples, 35 classes) and VoxCeleb1 (1,251 speakers). Scorer weights calibrated via joint verification grid search over 10×10 weight combinations + DET-driven threshold selection.
| Metric | Achieved | Target | Status |
|---|---|---|---|
| Parameters | 1.806 M | < 3.0 M | ✅ |
| ONNX Model Size | 0.60 MB (INT8) | — | ✅ |
| CPU Latency | 26.43 ms (p95: 28.29 ms) | < 200 ms | ✅ |
| Real-Time Factor (xRT) | 0.0132 | < 0.20 | ✅ |
| Keyword EER (standalone) | 4.69% | low | ✅ |
| Speaker EER (standalone) | 17.86% | low | ✅ |
| Joint EER | 23.47% | — | — |
| Joint AUC | 0.8425 | — | ✅ |
| Optimal Scorer Weights | wₖw=0.30, wₛₚₖ=0.65 | — | ✅ |
| EER Threshold (τ) | 0.2222 | — | ✅ |
License & Attribution
This project is licensed under the MIT License.
It transfers weights from and builds upon the open-source SpeechBrain repository (ECAPA-TDNN for speaker verification), licensed under Apache 2.0.
Team: Noisy AF
- Sohini Banerjee
- Swarnim Tripathi
VIT Chennai
- Library
- pytorch
- License
- mit
- Languages
- en
- Created
- Jun 18, 2026
- Updated
- Jun 21, 2026