Downloads · 30 days
0
Synthetic-Voice-Detection-Vocoder-Artifacts
csun22/Synthetic-Voice-Detection-Vocoder-Artifacts
Synthetic-Voice-Detection-Vocoder-Artifacts is a machine learning model from csun22. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
1. We are the first to identify neural vocoders as a source of features to expose synthetic human voices. Here are the differences shown by the six vocoders compared to the original audio:
Access
Public
Updated Apr 10, 2025
Repo size
75.5 MB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.pth74.8 MB · 98%
At a glance
- Access
- Public
- Created
- Apr 2, 2025
- Updated
- Apr 10, 2025
- SHA
- ca1888b7
Model card
From the Hugging Face model README
Synthetic-Voice-Detection-Vocoder-Artifacts
LibriSeVoc Dataset
-
We are the first to identify neural vocoders as a source of features to expose synthetic human voices. Here are the differences shown by the six vocoders compared to the original audio:
-
We provide LibriSeVoC as a dataset of self-vocoding samples created with six state-of-the-art vocoders to highlight and exploit the vocoder artifacts. The composition of the data set is shown in the following table: <img width="1000" alt="image" src="https://github.com/csun22/Synthetic-Voice-Detection-Vocoder-Artifacts/assets/90001788/c74fdb20-a5b7-4109-b833-821dd8dd6230"> The source of our dataset ground truth comes from LibriTTS. Therefore, we follow the naming logic of LibriTTS. For example, 27_123349_000006_000000.wav, 27 is the reader's ID, and 123349 is the ID of the chapter.
Deepfake Detection
We propose a new approach to detecting synthetic human voices by exposing signal artifacts left by neural vocoders and modifying and improving the RawNet2 baseline by adding multi-loss, lowering the error rate from 6.10% to 4.54% on the ASVspoof Dataset. This is the framework of the proposed synthesized voice detection method: <img width="1000" alt="image" src="https://github.com/csun22/Synthetic-Voice-Detection-Vocoder-Artifacts/assets/90001788/c46df06b-6d62-4b0f-a9d2-f5ffc4e378b9">
Paper & Dataset
For more details, please read our paper: https://openaccess.thecvf.com/content/CVPR2023W/WMF/html/Sun_AI-Synthesized_Voice_Detection_Using_Neural_Vocoder_Artifacts_CVPRW_2023_paper.html
For more details, please download our dataset: https://drive.google.com/file/d/1NXF9w0YxzVjIAwGm_9Ku7wfLHVbsT7aG/view
To train the model run:
python main.py --data_path /your/path/to/LibriSeVoc/ --model_save_path /your/path/to/models/
To test with your sample run:
python eval.py --input_path /your/path/to/sample.wav --model_path /your/path/to/your_model.pth
The weight of the trained model:
https://drive.google.com/file/d/1TWdsCFKP2luAfhpB91N9X4z1gsJMvvhI/view?usp=drive_link
In the wild testing:
Test on our Lab's Deepfake O Meter: https://zinc.cse.buffalo.edu/ubmdfl/deep-o-meter/landing_page
- Created
- Apr 2, 2025
- Updated
- Apr 10, 2025