video-sentiment-model

🎥 Multimodal Sentiment & Emotion Analysis Model

This model predicts sentiment (positive, neutral, negative) and emotion (joy, sadness, anger, etc.) from video data using audio, text, and visual features.
It was trained on the MELD dataset, which contains dialogue clips from the TV show Friends.

👉 Full training and deployment scripts are available here:
GitHub Repository – BryenInsights/multimodal-sentiment-emotion

---

🧾 Model Details

• Framework: PyTorch
• Input modalities:
  • Text: transcribed dialogue
  • Audio: speech features
  • Video: visual embeddings
• Output:
  • Sentiment: negative | neutral | positive
  • Emotion: anger | disgust | fear | joy | neutral | sadness | surprise

The version shared here is the normalized model (recommended for inference).

---

📊 Training Metrics

Training was run for 24 epochs. Loss curves show steady improvement.

You can find the full log of metrics in metrics.json.

---

🚀 Usage

You can download the model weights directly from this Hub repo and load them in PyTorch:

import torch

# Load model weights
state_dict = torch.load("model.pth", map_location="cpu")

# If using the provided model class (see GitHub repo)
from models import MultimodalSentimentModel
model = MultimodalSentimentModel(...)
model.load_state_dict(state_dict)
model.eval()

For full preprocessing pipeline (feature extraction, normalization, and inference), see the GitHub repo.

---

📌 Notes

•	Dataset (MELD.Raw.tar.gz) is not included here due to size; please download it directly from the MELD repo.
•	Only the normalized model is provided for easier use.

---

🙏 Acknowledgements

This project is adapted from the tutorial by Andreas Trolle.
My contribution was to reproduce the training pipeline, understand the design, and adapt it for deployment (e.g., AWS SageMaker).