voiceguard

<div align="center">

🛡️ VoiceGuard: AI-Powered Real-Time Voice Cloning Detection & Forensic Speech Biometrics

Problem Statement SIH26104: AI-Powered Real-Time Detection and Prevention of Voice Cloning Impersonation Attacks

<p align="center"> <b>VoiceGuard</b> is a production-grade, deepfake speech biometrics defense system engineered to detect, classify, and neutralize real-time voice cloning impersonation attacks across high-compression messaging platforms and cellular telephony networks. </p>

Key Features • Architecture • Quickstart • Forensic Telemetry • Training Protocol • Repository Layout

---

</div>

🌟 Key Capabilities

<table> <tr> <td width="50%"> <h3>🎯 Partial Transformer Unfreezing</h3> <p>Fine-tuned on <b>Wav2Vec 2.0</b> with permanently frozen 7-layer temporal CNNs and lower transformers, unfreezing strictly the top 2–3 layers to isolate neural vocoder phase artifacts without catastrophic forgetting.</p> </td> <td width="50%"> <h3>📡 Anti-Lossy Codec Robustness</h3> <p>Hardened against real-world messaging compression (<b>WhatsApp OGG, Telegram AAC, MP3, Opus</b>) and 300–3400 Hz G.711/AMR-NB telephony bandpass degradation to virtually eliminate false alarms.</p> </td> </tr> <tr> <td width="50%"> <h3>🌐 Dual-Track Indic & English Defense</h3> <p>Trained and benchmarked across multilingual speech corpuses: <b>English</b> (ASVspoof 2019/2021 LA + LibriSpeech) and <b>Indic</b> (IndicSuperb / IndicSynth Hindi & Tamil cloned speech).</p> </td> <td width="50%"> <h3>⚡ Active Kill-Switch & Low Latency</h3> <p>Sub-50ms CPU/GPU inference latency with calibrated temperature scaling, triggerable kill-switch disconnect threshold (75%), and 256-dim L2-normalized forensic embeddings.</p> </td> </tr> <tr> <td width="50%"> <h3>🔬 Deep Forensic Acoustic Metrics</h3> <p>Real-time physical signal diagnostics: <b>SNR</b>, <b>Zero-Crossing Rate</b>, <b>Spectral Centroid</b>, <b>Spectral/Pitch Flatness</b>, and <b>Clipping Ratio</b> with forensic anomaly tag generation.</p> </td> <td width="50%"> <h3>🧬 Multi-Task Vocoder Attribution</h3> <p>Simultaneously predicts binary threat scores via <b>AM-Softmax</b> ($s=30, m=0.35$) while attributing synthetic speech to vocoder families (<code>diffwave</code>, <code>melgan</code>, <code>wavenet</code>, <code>elevenlabs_neural</code>).</p> </td> </tr> </table>

---

🏗️ Model Architecture

The core pipeline processes 3.0-second raw audio windows ($48{,}000$ samples @ 16 kHz) through a multi-stage acoustic transformer:

flowchart TD
    A["Raw Audio Input<br/>(3.0s @ 16 kHz PCM)"] --> B["Anti-Lossy Augmenter<br/>(MP3/OGG/AAC/G.711 Codec Degradation)"]
    B --> C["7-Layer Temporal CNN Feature Extractor<br/>(Permanently Frozen)"]
    C --> D["Wav2Vec 2.0 Transformer Layers 0–20<br/>(Frozen Phonetic Representations)"]
    D --> E["Wav2Vec 2.0 Transformer Layers 21–23 + LayerNorm<br/>(Trainable: Vocoder Phase Artifact Adaptation)"]
    E --> F["Statistical Attention Pooling (SAP)<br/>Mean + Std Temporal Aggregation (2048-dim)"]
    F --> G["Forensic Projection Block<br/>Dense -> BN -> LeakyReLU -> Dropout -> Dense -> BN"]
    G --> H["256-dim L2-Normalized Forensic Embedding"]
    H --> I["Additive Margin Softmax (AM-Softmax)<br/>s=30.0, m=0.35 Angular Hypersphere Separation"]
    H --> J["Primary Binary Threat Head<br/>Real vs. Fake (Temperature Calibrated)"]
    H --> K["Auxiliary Vocoder Head<br/>[bona_fide, diffwave, melgan, wavenet, elevenlabs]"]
    J --> L{"Kill-Switch Engine<br/>Threshold >= 0.75"}
    L -->|"Fake Detected"| M["🛑 KILL_SWITCH_BLOCKED<br/>(Auto Terminate Call)"]
    L -->|"Authentic"| N["✅ AUTHENTIC_VERIFIED<br/>(Pass Audio)"]

Parameter Allocation Breakdown

---

⚡ Quickstart: 3-Minute Inference

Installation

git clone https://huggingface.co/indrajit4533/voiceguard
cd voiceguard
pip install torch torchaudio transformers soundfile scikit-learn numpy

1. Python Inference API

import torch
from voiceguard_wav2vec2 import VoiceGuardWav2Vec2
from inference import VoiceGuardInferenceEngine

# Initialize the production inference engine with calibrated kill-switch threshold
engine = VoiceGuardInferenceEngine(
    checkpoint_path="checkpoints/best_voiceguard_wav2vec2.pt", # optional
    threshold=0.75,
    temperature=1.0
)

# Inspect an incoming suspicious audio stream or file
report = engine.inspect_audio("suspicious_call.wav", return_embedding=False)

print(f"Decision:     {report['decision']}")
print(f"Threat Score: {report['threat_score'] * 100:.2f}% ({report['threat_level']})")
print(f"Vocoder Type: {report['vocoder_analysis']['predicted_subtype']}")
print(f"Anomaly Tags: {report['forensic_signal_metrics']['anomaly_tags']}")

2. Sample Production JSON Biometrics Output

{
    "decision": "KILL_SWITCH_BLOCKED",
    "threat_score": 0.9421,
    "threat_level": "CRITICAL",
    "threshold_applied": 0.75,
    "probabilities": {
        "bona_fide": 0.0579,
        "spoof_clone": 0.9421
    },
    "vocoder_analysis": {
        "predicted_subtype": "elevenlabs_neural",
        "confidence": 0.8874
    },
    "forensic_signal_metrics": {
        "snr_db": 14.8,
        "zero_crossing_rate": 0.0842,
        "spectral_centroid_hz": 1820.5,
        "spectral_flatness": 0.4921,
        "rms_energy": 0.2814,
        "clipping_ratio": 0.00012,
        "anomaly_tags": [
            "VOC_PHASE_INCOHERENCE",
            "TELEPHONY_COMPRESSION"
        ]
    },
    "latency_ms": 41.8
}

---

🔬 Forensic Acoustic Metrics

VoiceGuard couples deep transformer representations with deterministic digital signal processing (DSP) forensics:

Anomaly Flagging Tags

• VOC_PHASE_INCOHERENCE: Severe spectral flatness degradation characteristic of neural vocoders.
• TELEPHONY_COMPRESSION: Frequency bandwidth constrained within narrowband telecom limits (300–3400 Hz).
• CLIPPED_PEAKS: Digital amplifier clipping exceeding safety margin (> 0.5%).
• PITCH_FLATNESS: Unnaturally monotonous pitch contour found in early generation synthesis.
• LOW_SNR_DEGRADATION: Poor signal quality degrading downstream classification confidence.

---

🎯 Training & Optimization Protocol

1. Multi-Task Objective Loss Formulation

$$\mathcal{L}{\text{total}} = \mathcal{L}{\text{AM-Softmax}}(\text{binary}, s=30, m=0.35) + 0.3 \times \mathcal{L}_{\text{CE}}(\text{vocoder})$$

• Additive Margin Softmax: Enforces tight angular clustering of bona fide speakers on the 256-dimensional unit hypersphere while forcing synthetic cloned voice embeddings across an angular margin separation of $m=0.35$ with scale $s=30.0$.
• Vocoder Cross-Entropy: Trains the auxiliary head to identify vocoder fingerprints (bona_fide, diffwave, melgan, wavenet, elevenlabs_neural).

2. Differential Optimization Parameters

• Optimizer: AdamW ($\beta_1=0.9, \beta_2=0.98, \text{weight_decay}=10^{-4}$)
• Learning Rates:
  • Transformer Backbone: $\eta_{\text{backbone}} = 1 \times 10^{-5}$
  • Projection & Classification Heads: $\eta_{\text{heads}} = 1 \times 10^{-4}$
• Schedule: Linear warmup over the first 10% of total training steps, followed by Cosine Annealing decay.
• Stopping Criterion: Equal Error Rate (EER) minimization on the validation set with 3-epoch patience.

3. Launching Training

# Run training on custom manifest or built-in smoke-test dataset
python train.py --epochs 12 --batch_size 16 --output_dir checkpoints/

---

📁 Repository Structure

├── voiceguard_wav2vec2.py     # Core model, Statistical Pooling, AMSoftmaxLoss, LossyAugmenter
├── dataset.py                 # Dual-track Indic & English data loader + on-the-fly augmentation
├── train.py                   # Production training engine with differential LR & EER tracking
├── inference.py               # Real-time threat detection server & CLI biometrics reporter
├── push_to_hf.py              # Automated deployment script for Hugging Face Hub
├── README.md                  # Comprehensive Model Card & Technical Documentation
├── requirements.txt           # Verified package dependencies
└── checkpoints/               # Directory for best serialized PyTorch weights (.pt)

---

📊 Benchmarks

---

📜 License & Acknowledgments

This project is licensed under the MIT License.
Developed for Smart India Hackathon (Problem Statement SIH26104): AI-Powered Real-Time Detection and Prevention of Voice Cloning Impersonation Attacks.

Special thanks to the open-source speech communities behind ASVspoof, IndicSuperb / AI4Bharat, and Hugging Face Transformers.