ser-spanish-model-v7

Speech Emotion Recognition (SER) in Spanish V7

Model Details

Model Description

This SER model is trained on Mexican and Chilean audios dataset and finetuned from wav2vec2-xls-r-300m model

The possible emotions are:

• neutral
• sad
• disgusted
• fear
• anger
• happiness

Recommendations

• More data: search for additional audio samples from Chile or other Spanish-speaking countries
• Audio data augmentation: RIR convolution (reverb), SpecAugment on features
• Alternative models: facebook/wav2vec2-xls-r-1b (larger) or perhaps openai/whisper with a classifier
• Ensemble: combine predictions from multiple models
• Cross-validation: with such limited data, k-fold provides more robust estimates
• Error analysis: identify which pairs of emotions are most frequently confused

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import pipeline

model_id = "pollitoconpapass/ser-spanish-model-v7"
classifier = pipeline("audio-classification", model=model_id, device=0 if torch.cuda.is_available() else -1)

# Audio de prueba del dataset
test_audio_path = "wav_file_path.wav"

# Prediccion
prediction = classifier(test_audio_path)

print("\nResultados del modelo:")
for res in prediction:
    print(f" - {res['label']}: {res['score']:.1%}")

Training Details

Training Data

• Kaggle Dataset for Mexican Audio: saurabhshahane/mexican-emotional-speech-database-mesd
• Kaggle Dataset for Chilean Audio: josequispezav/audio-emociones-chile

Training Hyperparameters

training_args = TrainingArguments(
    output_dir=OUTPUT_DIR,

    # Epochs y batch (effective batch = 32)
    num_train_epochs=num_epochs,
    per_device_train_batch_size=batch_size,
    per_device_eval_batch_size=8,
    gradient_accumulation_steps=gradient_accumulation_steps,

    # Optimizador
    learning_rate=3e-5,
    warmup_steps=warmup_steps, # -> warmup_steps en lugar de warmup_ratio
    weight_decay=0.01,
    lr_scheduler_type='cosine', # -> cosine despues del decay

    # Evaluar y guardar
    eval_strategy='epoch',
    save_strategy='epoch',
    save_total_limit=1,  # -> limita checkpoints activos para ahorrar espacio
    load_best_model_at_end=True,
    metric_for_best_model='macro_f1', # -> criterio principal
    greater_is_better=True,

    # Adicional
    dataloader_drop_last=True,

    # GPU
    fp16=torch.cuda.is_available(),
    dataloader_num_workers=2,

    # Loggers
    logging_steps=10,
    report_to='none',

    seed=SEED,
)

trainer = WeightedTrainer(
    class_weights=class_weights,
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    compute_metrics=compute_metrics,
    callbacks=[
        EarlyStoppingCallback(early_stopping_patience=5)  # Paciencia de solo 5 epochs (sino mejora en 5 epochs, se para el entrenamiento)
    ],
)

Metrics

• F1 macro

Results

============================================================
REPORTE DE CLASIFICACIÓN — TEST SET
============================================================
              precision    recall  f1-score   support

    disgusto       0.82      0.90      0.86        20
       enojo       0.87      0.54      0.67        24
       feliz       0.61      0.87      0.71        23
       miedo       0.60      0.14      0.23        21
     neutral       0.85      0.88      0.86        25
      triste       0.63      0.96      0.76        23

    accuracy                           0.72       136
   macro avg       0.73      0.72      0.68       136
weighted avg       0.73      0.72      0.69       136