w2v-speech-emotion-recognition

Wav2Vec2 Speech Emotion Recognition for English

Model Overview

This model is fine-tuned for recognizing emotions in English speech using the Wav2Vec2 architecture. It is capable of detecting the following emotions:

• Sadness
• Anger
• Disgust
• Fear
• Happiness
• Neutral The model was trained on the Speech Emotion Recognition dataset from Kaggle, which consists of English emotional speech samples. The dataset includes audio files labeled with various emotional states, making it ideal for training models in emotion recognition tasks.

Model Details

• Architecture: Wav2Vec2
• Languages: English
• Dataset: Speech Emotion Recognition Dataset (Kaggle)
• Emotions Detected: Sadness, Anger, Disgust, Fear, Happiness, Neutral

Training Results

The model achieved the following results on the test set:

Test Accuracy: 0.7435

Classification Report:

              precision    recall  f1-score   support

     sadness       0.68      0.71      0.70       251
       angry       0.75      0.93      0.83       258
     disgust       0.86      0.64      0.73       250
        fear       0.75      0.61      0.67       287
       happy       0.73      0.68      0.71       231
     neutral       0.72      0.92      0.81       212

    accuracy                           0.74      1489
   macro avg       0.75      0.75      0.74      1489
weighted avg       0.75      0.74      0.74      1489

How to Use

Installation

To use this model, you need to install the transformers and torchaudio packages:

pip install transformers
pip install torchaudio

Example Usage

Here is an example of how to use the model to classify emotions in an English audio file:

from transformers import pipeline

# Load the fine-tuned model and feature extractor
pipe = pipeline("audio-classification", model="Khoa/w2v-speech-emotion-recognition")

# Path to your audio file
audio_file = "path_to_your_audio_file.wav"

# Perform emotion classification
predictions = pipe(audio_file)

# Map predictions to real emotion labels
label_map = {
    "LABEL_0": "sadness",
    "LABEL_1": "angry",
    "LABEL_2": "disgust",
    "LABEL_3": "fear",
    "LABEL_4": "happy",
    "LABEL_5": "neutral"
}

# Convert predictions to readable labels
mapped_predictions = [
    {"score": pred["score"], "label": label_map[pred["label"]]} 
    for pred in predictions
]

# Display results
print(mapped_predictions)

Example Output

The model outputs a list of predictions with scores for each emotion. For example:

[
    {"score": 0.95, "label": "angry"},
    {"score": 0.02, "label": "happy"},
    {"score": 0.01, "label": "disgust"},
    {"score": 0.01, "label": "neutral"},
    {"score": 0.01, "label": "fear"}
]

Training Details

The model was fine-tuned on the Speech Emotion Recognition Dataset, using the Wav2Vec2 architecture. The training process involved multiple epochs with a learning rate of 1e-5.

Limitations and Biases

This model is specifically trained on English speech data and may not perform well on other languages or dialects. Additionally, as with any machine learning model, there may be biases present in the training data that could affect the model's predictions.