resnet50-emotion-recognition-ckplus-rafdb

🧠 Emotion Recognition Model – ResNet50 (Fine-Tuned on CK+ and RAF-DB)

📘 Overview

This repository presents a fine-tuned ResNet50-based Emotion Recognition model trained on the CK+ and RAF-DB facial expression datasets. The model classifies facial emotions into seven categories and provides high accuracy and generalization through a two-phase fine-tuning process. Both TensorFlow and TensorFlow Lite versions are available for deployment on cloud and edge devices.

This model forms a key component of an AI-powered, emotion-aware sign language translation system, enabling real-time emotion recognition and integration into affective computing pipelines.

---

🧩 Model Architecture

The architecture is built on ResNet50, pre-trained on ImageNet, and fine-tuned in two stages:

1. Stage 1 – Frozen Base Training (10 Epochs):

   • Convolutional layers frozen.
   • Only top dense layers trained to learn task-specific emotion representations.

2. Stage 2 – Unfrozen Base Fine-Tuning (30 Epochs):

   • Entire base unfrozen with a lower learning rate.
   • Enables deep adaptation to emotion-related facial features.

---

📊 Datasets

1. CK+ Dataset (Kaggle)
2. RAF-DB Dataset (Kaggle)

Dataset Preparation

• Combined CK+ and RAF-DB datasets to increase emotion diversity.
• Applied extensive data augmentation (rotation, flips, brightness, zoom, and shifts).
• Ensured balanced class distribution post-augmentation.

---

⚙️ Training Configuration

---

📈 Performance Summary

Classification Report

Overall Accuracy: 81.65% Weighted F1-Score: 0.81

---

🖼️ Visualizations

1. Training Accuracy and Loss

• Graph 1: Training and Validation Accuracy vs Epochs

• Graph 2: Training and Validation Loss vs Epochs

2. Dataset Distributions

• Graph 3: Original Dataset Class Distribution

• Graph 4: Balanced (Augmented) Dataset Class Distribution

3. Evaluation Visuals

• Graph 5: Multi-Class ROC Curves (AUC per class)

• Graph 6: Confusion Matrix (Heatmap)

• Graph 7: Sample Test Results (5 predictions per class)

These visualizations demonstrate consistent learning, balanced data, and strong class-wise recognition.

---

🧩 Model Files

---

🧰 Inference Example

import tensorflow as tf
from tensorflow.keras.preprocessing import image
import numpy as np

# Load original model
model_path = 'emotion_resnet50_model.h5'
model = tf.keras.models.load_model(model_path)

# Prepare input
img = image.load_img('test_face.jpg', target_size=(224, 224))
input_data = np.expand_dims(image.img_to_array(img) / 255.0, axis=0)

# Run inference
pred = model.predict(input_data)
classes = ['Angry', 'Disgust', 'Fear', 'Happy', 'Neutral', 'Sad', 'Surprise']
print("Original Model Prediction:", classes[np.argmax(pred)])

# Load and run TFLite optimized model
tflite_model_path = 'emotion_resnet50_optimized.tflite'
interpreter = tf.lite.Interpreter(model_path=tflite_model_path)
interpreter.allocate_tensors()
input_index = interpreter.get_input_details()[0]['index']
output_index = interpreter.get_output_details()[0]['index']
interpreter.set_tensor(input_index, input_data.astype(np.float32))
interpreter.invoke()
output = interpreter.get_tensor(output_index)
print("TFLite Model Prediction:", classes[np.argmax(output)])

---

🚀 Key Features

• Dual-dataset fine-tuning (CK+ + RAF-DB)
• Data-balanced training via augmentation
• High accuracy with stable generalization
• Lightweight TensorFlow Lite version for edge devices
• Ideal for integration in emotion-aware systems

---

🏷️ Tags

emotion-recognition resnet50 facial-expression deep-learning tensorflow tflite ckplus rafdb computer-vision affective-computing fine-tuning

---

📄 Citation

@misc{pasindu_sewmuthu_abewickrama_singhe_2025,
	author       = { Pasindu Sewmuthu Abewickrama Singhe },
	title        = { resnet50-emotion-recognition-ckplus-rafdb (Revision 3216038) },
	year         = 2025,
	url          = { https://huggingface.co/PSewmuthu/resnet50-emotion-recognition-ckplus-rafdb },
	doi          = { 10.57967/hf/6653 },
	publisher    = { Hugging Face }
}

---

👤 Author & Model Info

Author: P.S. Abewickrama Singhe
Developed with: TensorFlow + Keras
License: Apache-2.0
Date: October 2025
Email: [email protected]