persuasive_language_detector

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Given a sentence, our model predicts whether or not the sentence contains "persuasive" language, or language designed to elicit emotions or change readers' opinions. The model was tuned on the SemEval 2020 Task 11 dataset. However, we preprocessed the dataset to adapt it from multilabel technique classification and span-classification to our binary classification task.

There are two revisions:

• BERT - we finetuned bert-large-cased on our main branch
• XLM-RoBERTa - we finetuned xlm-roberta-base on our roberta branch.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Ultraviolet Text
• Model type: BERT / RoBERTa
• Language(s) (NLP): En
• License: MIT
• Finetuned from model [optional]: bert-large-cased / xlm-roberta-base

How to Get Started with the Model

Use the code below to get started with the model.

Loading from the main branch (BERT)

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("bert-large-cased")
model = AutoModelForSequenceClassification.from_pretrained("chreh/persuasive_language_detector")

Loading from the roberta branch (XLM RoBERTa)

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-base")
model = AutoModelForSequenceClassification.from_pretrained("chreh/persuasive_language_detector", revision="roberta")

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

Training data can be downloaded from the Semeval website.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

The training was done using Huggingface Trainer on both our local machines and Intel Developer Cloud kernels, enabling us to prototype multiple models simultaneously.

Preprocessing [optional]

All sentences containing spans of persuasive language techniques were labeled as persuasive language examples, while all others were labeled as examples of non-persuasive language.

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

The test data is from the test data of sem_eval_2020_task_11, which can be downloaded from the original website. The test data contains 38.25% persuasive examples and non-persuasive examples 61.75%. Metrics can be found in the following section

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Metrics are reported in the format (main_branch), (roberta branch)

• Accuracy - 0.7165140725669719, 0.7326693227091633
• Recall - 0.6875584658559402, 0.6822916666666666
• Precision - 0.5941794664510913, 0.6415279138099902
• F1 - 0.6374674761491761, 0.6612821807168097

Overall, the roberta branch performs better, and with faster inference times. Thus, we recommend users download from the roberta revision.