xlm-roberta-argument

Welcome to XLM-RoBERTArgument!

🤖 Model description

XLM-RoBERTArgument is a model trained by the Hyperloop UPV team (https://hyperloopupv.com/) in order to detect arguments. It is part of the competition of the European Hyperloop Week (https://hyperloopweek.com/). This model was trained on ~50k heterogeneous manually annotated sentences (📚 Stab et al. 2018 and the machine-translation to Spanish) of controversial topics to classify text into one of two labels: 🏷 NON-ARGUMENT (0) and ARGUMENT (1).

🗃 Dataset

The dataset (📚 Stab et al. 2018) consists of ARGUMENTS (~11k for each language) that either support or oppose a topic if it includes a relevant reason for supporting or opposing the topic, or as a NON-ARGUMENT (~14k for each language) if it does not include reasons. The authors focus on controversial topics, i.e., topics that include "an obvious polarity to the possible outcomes" and compile a final set of eight controversial topics: abortion, school uniforms, death penalty, marijuana legalization, nuclear energy, cloning, gun control, and minimum wage. The following table shows the number of argument and non-arguments for each language depending on the topic.

🏃🏼‍♂️Model training

XLM-RoBERTArgument was fine-tuned on a XLM-RoBERTA (large) pre-trained model from HuggingFace using the a custom trainer with the following hyperparameters:

num_train_epochs=3
learning_rate=1e-06
per_device_train_batch_size=8
per_device_eval_batch_size=8

📊 Evaluation

The model was evaluated on an evaluation set (20%):

👀 Usage with PyTorch model

import torch 
from torch import nn
import numpy as np
from transformers import AutoTokenizer, XLMRobertaModel
from XLMRobertargument import XLMRoBERTaClassifier # Import classifier

# Load model
model= torch.load("xlmrobertargument.pt")

tokenizer = AutoTokenizer.from_pretrained('xlm-roberta-large')
tok = tokenizer("Hola cómo estás",padding='max_length', max_length = 200,
                      truncation=True, return_tensors="pt")
out = model(torch.Tensor(tok["input_ids"]).long(), torch.LongTensor(tok["attention_mask"]).long())
print(out)

# tensor(0.0171, grad_fn=<SqueezeBackward0>) 

⚠️ Intended Uses & Potential Limitations

The model can only be a starting point to dive into the exciting field of argument mining. But be aware. An argument is a complex structure, with multiple dependencies. Therefore, the model may perform less well on different topics and text types not included in the training set.