voicemail-detector-wav2vec2-onnx

Voicemail Detector - Wav2Vec2 (ONNX FP16)

This model detects whether an audio signal contains a live human voice or a voicemail greeting. It's based on fine-tuned Wav2Vec2 and exported to ONNX format with FP16 precision for efficient inference.

Model Description

• Model Type: Audio Classification (Binary)
• Base Model: jakeBland/wav2vec-vm-finetune
• Format: ONNX with external data (FP16)
• Input: 2 seconds of audio at 16kHz (32,000 samples)
• Output: Binary classification (live_human vs voicemail)
• Model Size: ~605 MB (FP16 optimized)

Performance Metrics

Accuracy

• Overall Accuracy: 72.73%
• Live Human Detection: 100% (3/3)
• Voicemail Detection: 62.5% (5/8)

Inference Speed

• Average Inference Time: 705ms (CPU)
• Memory Usage: ~1.6 GB during inference

Comparison with CNN Model

While the CNN model is faster (11ms) and smaller (18MB), the Wav2Vec2 model excels at:

• Perfect live human detection (100% vs 67%)
• Better on complex voicemail scenarios with realistic speech patterns

Use Cases

This model is ideal for:

• 📞 Automated phone systems that need to distinguish human responses from voicemail greetings
• 🎯 Call centers prioritizing live human connections
• 🤖 Voice assistants that need to detect voicemail before speaking
• 📊 Analytics systems tracking call connection rates

Best suited for: Applications where minimizing false positives on live humans is critical, and latency tolerance is >500ms.

Installation

pip install onnxruntime transformers numpy

Usage

Basic Inference

import numpy as np
import onnxruntime as ort
from transformers import Wav2Vec2FeatureExtractor

# Load the feature extractor (critical for proper preprocessing!)
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained(
    "jakeBland/wav2vec-vm-finetune"
)

# Load ONNX model
session = ort.InferenceSession("model.onnx")

# Prepare audio (2 seconds at 16kHz = 32,000 samples)
# audio_array should be a numpy array of shape (32000,)
inputs = feature_extractor(
    audio_array,
    sampling_rate=16000,
    return_tensors="np",
    padding=True,
)

# Run inference
input_values = inputs.input_values.astype(np.float32)
outputs = session.run(None, {"input": input_values})
logits = outputs[0]

# Get prediction
prediction_idx = np.argmax(logits, axis=-1)[0]
result = "voicemail" if prediction_idx == 1 else "live_human"
print(f"Detection result: {result}")

With Audio File

import librosa
import numpy as np

# Load audio file (mono, 16kHz)
audio, sr = librosa.load("audio.wav", sr=16000, mono=True)

# Take first 2 seconds (32,000 samples)
audio_segment = audio[:32000]

# Pad if shorter than 2 seconds
if len(audio_segment) < 32000:
    audio_segment = np.pad(audio_segment, (0, 32000 - len(audio_segment)))

# Now use the inference code above

Important Implementation Notes

⚠️ Critical: Feature Extraction is Required

This model requires proper preprocessing using Wav2Vec2FeatureExtractor. Without normalization, accuracy drops from 73% to 36%.

The feature extractor applies:

• Mean-std normalization (per-sample)
• Padding configuration
• Attention mask generation

DO NOT skip this step or implement custom normalization. Always use the official transformer's feature extractor.

Audio Requirements

• Duration: Exactly 2 seconds (32,000 samples)
• Sample Rate: 16kHz
• Channels: Mono
• Format: Float32 numpy array
• Normalization: Via Wav2Vec2FeatureExtractor (do_normalize=True)

Model Input/Output

Input:

• Name: input
• Shape: [1, 32000]
• Type: float32
• Range: Normalized by feature extractor (mean=0, std=1)

Output:

• Name: logits
• Shape: [1, 2]
• Type: float32
• Classes: [0: live_human, 1: voicemail]

Training Details

• Original Model: jakeBland/wav2vec-vm-finetune
• Fine-tuning Task: Binary classification for voicemail detection
• Export Method: PyTorch → ONNX with FP16 quantization
• Optimization: External data format for large models

Limitations

• Speed: 705ms inference time (65x slower than CNN alternative)
• Memory: Requires ~1.6GB RAM during inference
• Deployment: Larger model size requires careful resource planning
• Edge Cases: Struggles with simple beeps and silent audio (prefers the CNN model for these)

Evaluation Results

Tested on 11 audio samples:

Failed Cases:

• Simple beep tones (beep.mp3, beep_2.mp3)
• Silent audio (silent.mp3)

Citation

If you use this model, please cite the original Wav2Vec2 voicemail detector:

@misc{wav2vec-vm-finetune,
  author = {Jake Bland},
  title = {Wav2Vec2 Voicemail Fine-tune},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/jakeBland/wav2vec-vm-finetune}
}

License

This model follows the license of the original jakeBland/wav2vec-vm-finetune model.

Model Card Contact

For questions or issues with this ONNX export, please open an issue in the repository.

---

Related Models:

• CNN Voicemail Detector - Faster, smaller alternative (11ms, 18MB)
• Original PyTorch Model