whisper-deepfake-detector

Whisper-based Deepfake Audio Detector

This model is a fine-tuned Whisper model for binary classification of real vs fake (deepfake) audio.

Model Details

• Base Model: openai/whisper-base
• Task: Binary Classification (Real vs Fake Audio)
• Sample Rate: 16000 Hz
• Audio Duration: 2.0 seconds
• Test Accuracy: 99.94%

Training Details

• Training Samples: 12509
• Validation Samples: 1787
• Test Samples: 3574
• Epochs: 5
• Learning Rate: 1e-05
• Batch Size: 16

Usage

Quick Start - Easy Loading

The model is saved in safetensors format for easy and safe loading. Use the provided utilities:

Option 1: Using the Loading Utility (Recommended)

from load_model import load_whisper_classifier
import librosa
import numpy as np
import torch

# Load model (one line!)
model, processor = load_whisper_classifier(
    repo_id="Akshay-Sai/whisper-deepfake-detector",
    device="cpu"  # or "cuda" if GPU available
)

# Preprocess audio (16kHz, 2 seconds)
audio, sr = librosa.load("audio.wav", sr=16000, duration=2.0)
target_length = int(16000 * 2.0)
if len(audio) < target_length:
    audio = np.pad(audio, (0, target_length - len(audio)), mode='constant')
elif len(audio) > target_length:
    audio = audio[:target_length]

# Process and predict
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
input_features = inputs.input_features

with torch.no_grad():
    logits = model(input_features)
    probabilities = torch.softmax(logits, dim=-1)
    predicted_class = logits.argmax(dim=-1).item()

print(f"Prediction: {'REAL' if predicted_class == 0 else 'FAKE'}")

Option 2: Using Command Line Inference Script

python inference.py --audio_path "path/to/audio.wav" --repo_id "Akshay-Sai/whisper-deepfake-detector"

Option 3: Manual Loading

from transformers import WhisperProcessor
from modeling_whisper_classifier import WhisperClassifier
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
import torch

# Load model architecture
base_model_path = "openai/whisper-base"
model = WhisperClassifier(base_model_path, num_classes=2)

# Load weights from safetensors
weights_path = hf_hub_download(repo_id="Akshay-Sai/whisper-deepfake-detector", filename="model.safetensors")
state_dict = load_file(weights_path)
model.load_state_dict(state_dict)
model.eval()

# Load processor
processor = WhisperProcessor.from_pretrained(base_model_path)

Results

• Best Validation Accuracy: 99.94%
• Test Accuracy: 99.94%
• Overfit Gap: 0.06%

Limitations

This model was trained on a specific dataset and may not generalize to all types of deepfake audio. Performance may vary on different datasets.

Citation

If you use this model, please cite:

@misc{whisper-deepfake-detector,
  title={Whisper-based Deepfake Audio Detector},
  author={Your Name},
  year={2024},
  publisher={HuggingFace},
  howpublished={\url{https://huggingface.co/Akshay-Sai/whisper-deepfake-detector}}
}