bert-emotion

Sentiment Analysis with BERT

This BERT-based model for sentiment analysis was created by me as a student completing Vanderbilt Data Science Institute's AI Summer Course in 2023. It serves as an introductory example of fine-tuning a pretrained model for a downstream task.

What I Learned

• Leveraging transfer learning instead of training a model from scratch
• Fine-tuning a pretrained model on a downstream dataset
• Implementing optimizations like learning rate scheduling
• Evaluating models using relevant metrics like accuracy

About the Project

During the program, I explored various techniques for adapting powerful large-scale models like BERT to specialized applications. As a hands-on exercise, I fine-tuned BERT using the tweet_eval dataset to classify text snippets as either positive or negative in sentiment.

This model is the result of that exercise, providing my basic implementation of sentiment classification using BERT fine-tuning. While not as performant as state-of-the-art sentiment models, it demonstrates the workflow and techniques I learned around tailoring BERT and similar models.

The training code is provided to allow replication and customization for other datasets. I hope this model provides a useful case study for anyone beginning their journey into fine-tuning and transfer learning with transformer models like I was!