roberta-base-sentiment-politics

eevvgg/sentimenTw-political

This model is a fine-tuned version of multilingual model cardiffnlp/twitter-xlm-roberta-base-sentiment. Classification of text sentiment into 3 categories: negative, neutral, positive. Fine-tuned on a 2k sample of manually annotated Reddit (EN) and Twitter (PL) data.

• Developed by: Ewelina Gajewska as a part of ComPathos project: https://www.ncn.gov.pl/sites/default/files/listy-rankingowe/2020-09-30apsv2/streszczenia/497124-en.pdf

• Model type: RoBERTa for sentiment classification

• Language(s) (NLP): Multilingual; finetuned on 1k English text from Reddit and 1k Polish tweets

• License: [More Information Needed]

• Finetuned from model: cardiffnlp/twitter-xlm-roberta-base-sentiment

Uses

Sentiment classification in multilingual data. Fine-tuned on a 2k English and Polish sample of social media texts from political domain. Model suited for short text (up to 200 tokens) .

How to Get Started with the Model

from transformers import pipeline

model_path = "eevvgg/sentimenTw-political"
sentiment_task = pipeline(task = "text-classification", model = model_path, tokenizer = model_path)

sequence = ["TRUMP needs undecided voters",
            "Oczywiście ze Pan Prezydent to nasza duma narodowa!!"]
            
result = sentiment_task(sequence)
labels = [i['label'] for i in result] # ['neutral', 'positive']            

Model Sources

• Repository: Colab notebook
• Paper: TBA
• BibTex citation:

@misc{SentimenTwGK2023,
  author={Gajewska, Ewelina and Konat, Barbara},
  title={SentimenTw XLM-RoBERTa-base Model for Multilingual Sentiment Classification on Social Media},
  year={2023},
  howpublished = {\url{https://huggingface.co/eevvgg/sentimenTw-political}},
}

Training Details

• Trained for 3 epochs, mini-batch size of 8.
• Training results: loss: 0.515
• See details in Colab notebook

Preprocessing

• Hyperlinks and user mentions (@) normalization to "http" and "@user" tokens, respectively. Removal of extra spaces.

Speeds, Sizes, Times

• See Colab notebook

Evaluation

• Evaluation run on a sample of 200 texts (10% of data).

Results

• accuracy: 74.0
• macro avg:
  • f1: 71.2
  • precision: 72.8
  • recall: 70.8
• weighted avg:

  • f1: 73.3

  • precision: 74.0

  • recall: 74.0

                   precision    recall  f1-score   support
    
       negative      0.752     0.901     0.820        91
       neutral       0.764     0.592     0.667        71
       positive      0.667     0.632     0.649        38
    

Citation

BibTeX:

@misc{SentimenTwGK2023,
  author={Gajewska, Ewelina and Konat, Barbara},
  title={SentimenTw XLM-RoBERTa-base Model for Multilingual Sentiment Classification on Social Media},
  year={2023},
  howpublished = {\url{https://huggingface.co/eevvgg/sentimenTw-political}},
}

APA:

Gajewska, E., & Konat, B. (2023).
SentimenTw XLM-RoBERTa-base Model for Multilingual Sentiment Classification on Social Media.
https://huggingface.co/eevvgg/sentimenTw-political.