XT-Sentiment-XLM-Roberta-Large

General description of the model

Unlike a classical sentiment classifier, this model was built to measure the sentiment towards a particular entity on a particular pre-determined topic

model = ....

text =  "I pity Facebook for their lack of commitment against global warming , I like google for its support of increased education"
# In the previous example we notice that depending on the type of entity (Google or Facebook) and depending on the type of to#pics (education or climate change) we have two types of sentiments

# Predict the sentiment towards Facebook (entity) on Climate change (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# sentiment = "negative

# Predict the sentiment towards Google (entity) on Education (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# Sentiment = "positive"

# Predict the sentiment towards Google (entity) on Climate Change (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# Sentiment = "neutral" / "not_found"

# Predict the sentiment towards Facebook (entity) on Education (topic) 
sentiment, probability = model.predict(text, topic="climate change", entity= "Facebook")
# Sentiment = "neutral" / "not_found"

Training

This is a SetFit model that can be used for sentiment classification. The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
3. The Training data can be downloaded from here

Usage and Inference

For a global overview of the pipeline used for inference please refer to this colab notebook

Model Performance

The performances of the model on our internal test set are:

• Accuracy: 0.68
• Balanced_Accuracy: 0.45
• MCC: 0.37
• F1: 0.49

Potential weakness of the model

• As the model has been trained on data of short length, it is difficult to predict how the model will behave on long texts
• Although the model is robust to typos and able to deal with synonyms, the entities and topics must be as explicit as possible.
• The model may have difficulties to detect very abstract and complex topics, a fine tuning of the model can solve this problem
• The model may have difficulty in capturing elements that are very specific to a given context

BibTeX entry and citation info

author = {HasiMichael, Solofo, Bruce, Sitwala},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Sentiment Classification toward Entity and Topics},
year = {2023/04},
version = {0}