Downloads · 30 days
52
39% of all-time downloads
huyleit/mert-v1-95m-music-emotion-int8
mert-v1-95m-music-emotion-int8 is a audio classification model from huyleit. Use it for the audio classification task on the model card, and read the license before you ship it in a product. It is set up for onnx. The card lists the license as apache-2.0.
Model Architecture: m-a-p/MERT-v1-95M (Music Foundation Transformer, ICLR 2024) + Temporal Attention Pooling → Dynamic Quantized INT8 ONNX Target Task: Continuous 2D Valence & Arousal (VA) Prediction in Russell's Circ…
Downloads · 30 days
52
39% of all-time downloads
All-time downloads
133
Public
Repo size
95.6 MB
Likes
2
Public
Click a slice to open those files.
.onnx95.6 MB · 100%
From the Hugging Face model README
Model Architecture:
m-a-p/MERT-v1-95M(Music Foundation Transformer, ICLR 2024) + Temporal Attention Pooling → Dynamic Quantized INT8 ONNX
Target Task: Continuous 2D Valence & Arousal (VA) Prediction in Russell's Circumplex Model[0.0, 1.0]
Optimized For: Ultra-fast CPU Inference, Light Memory Footprint (~111 MB RAM), Serverless / Microservice Deployment
This model is a specialized Music Emotion Recognition (MER) regression engine developed for affective computing and mental health music therapy recommendation systems (part of the SE121-microservices project).
It predicts continuous 2D coordinates:
Neutral Center: (0.5, 0.5)
| Quadrant | Valence (V) | Arousal (A) | Target Mood / Emotional States | Therapeutic Application |
|---|---|---|---|---|
| Q1 | ≥ 0.5 | ≥ 0.5 | Happy, Excited, Euphoric, Joyful, Upbeat | Focus, Motivation, Mood Lifting |
| Q2 | < 0.5 | ≥ 0.5 | Angry, Tense, Anxious, Frustrated, Distress | Catharsis, Tension Release |
| Q3 | < 0.5 | < 0.5 | Sad, Depressed, Melancholic, Gloomy | Empathy, Sleep Aid, Mild Soothing |
| Q4 | ≥ 0.5 | < 0.5 | Relaxed, Peaceful, Serene, Calm, Chill | Stress Reduction, Meditation |
Evaluated independently on a Held-Out Test Set of 386 songs (isolated 15% split from DEAM and PMEmo datasets, total N = 2,569 tracks):
======================================================================
EVALUATION ON HELD-OUT TEST SET (386 TRACKS)
======================================================================
Valence CCC (Lin's Correlation) : 0.7358 (Approaching Human Agreement Ceiling)
Valence R² Score : 0.4783
Valence MAE (Mean Absolute Err) : 0.0895
----------------------------------------------------------------------
Arousal CCC (Lin's Correlation) : 0.8204 (SOTA Performance)
Arousal R² Score : 0.6575
Arousal MAE (Mean Absolute Err) : 0.0861
======================================================================
[T/2 - 7.5s, T/2 + 7.5s]), leveraging the Thin-Slicing Paradigm and Chorus Salience Principle.[1, 360000] float32 tensor (24,000 × 15 = 360,000 samples).audio_waveformvalence_arousal (Shape: [1, 2])pip install onnxruntime soundfile scipy numpy huggingface_hub
import onnxruntime as ort
import soundfile as sf
import scipy.signal
import numpy as np
from huggingface_hub import hf_hub_download
# 1. Download model weights from Hugging Face Hub
model_path = hf_hub_download(
repo_id="huyleit/mert-v1-95m-music-emotion-int8",
filename="mert_emotion_int8.onnx"
)
# 2. Initialize ONNX Runtime Session
opts = ort.SessionOptions()
opts.intra_op_num_threads = 4
opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
session = ort.InferenceSession(model_path, sess_options=opts, providers=['CPUExecutionProvider'])
def predict_music_emotion(audio_path: str) -> dict:
target_sr = 24000
duration_sec = 15
target_samples = target_sr * duration_sec
# Read audio
data, sr = sf.read(audio_path, dtype='float32')
if data.ndim > 1:
data = np.mean(data, axis=1) # Mono channel
# Extract 15s center segment
orig_samples = int(sr * duration_sec)
if len(data) > orig_samples:
start = (len(data) - orig_samples) // 2
data = data[start:start + orig_samples]
elif len(data) < orig_samples:
data = np.pad(data, (0, orig_samples - len(data)))
# Resample to 24kHz
if sr != target_sr:
data = scipy.signal.resample(data, target_samples).astype(np.float32)
input_tensor = np.expand_dims(data, axis=0).astype(np.float32)
# Inference
outputs = session.run(["valence_arousal"], {"audio_waveform": input_tensor})
valence, arousal = float(outputs[0][0][0]), float(outputs[0][0][1])
# Classify Russell Quadrant
quadrant = "Q1" if valence >= 0.5 and arousal >= 0.5 else \
"Q2" if valence < 0.5 and arousal >= 0.5 else \
"Q3" if valence < 0.5 and arousal < 0.5 else "Q4"
return {
"valence": round(valence, 4),
"arousal": round(arousal, 4),
"quadrant": quadrant
}
# Example usage:
# result = predict_music_emotion("sample_song.mp3")
# print(result)
# Output: {'valence': 0.6011, 'arousal': 0.5947, 'quadrant': 'Q1'}
m-a-p/MERT-v1-95MLoss_total = 0.75 * Loss_CCC + 0.25 * Loss_MSE
@inproceedings{li2024mert,
title={MERT: Acoustic Music Understanding with Large-Scale Self-supervised Training},
author={Li, Yizhi and Yuan, Ruibin and Zhang, Ge and Ma, Yinghao and Lin, Chenghua and Chen, Xingran and Ragni, Anton and Benetos, Emmanouil and Gyenge, Norbert and Dannenberg, Roger and others},
booktitle={International Conference on Learning Representations (ICLR)},
year={2024}
}
@article{russell1980circumplex,
title={A circumplex model of affect.},
author={Russell, James A},
journal={Journal of personality and social psychology},
volume={39},
number={6},
pages={1161},
year={1980}
}