xlm-roberta-large-xnli

XLM-RoBERTa Large finetuned on XNLI dataset

<!-- Provide a quick summary of what the model is/does. -->

Model Details

• Developed by: Adrien J.
• Model type: XLM-RoBERTa
• Languages (NLP): Multilingual
• Finetuned from model: FacebookAI/xlm-roberta-large

How to Get Started

This model is ready-to-use for text classification.

import pandas as pd
from transformers import pipeline

# Load the classification pipeline
classifier = pipeline("text-classification", "ajayat/xlm-roberta-large-xnli")
classifier.model.config.id2label = {
    0: "entailment", 
    1: "neutral", 
    2: "contradiction"
}
# Example premise and hypothesis
premise = "A soccer game with multiple males playing."
hypothesis = "Some men are playing a sport."

# Provide input as a dictionary with text and text_pair keys
result = classifier({'text': premise, 'text_pair': hypothesis}, top_k=None)
pd.DataFrame(result)

Dataset

The XNLI dataset (Cross-lingual Natural Language Inference) is a benchmark dataset created by Facebook AI for evaluating cross-lingual understanding. It extends the MultiNLI corpus by translating 7,500 human-annotated English sentence pairs (premise and hypothesis) into 14 languages.

Each pair is labeled as entailment, contradiction, or neutral.

Training Hyperparameters

• bf16 mixed precision
• Batch size per GPU: 64
• Learning Rate: 2e-5
• 1 Epoch

Results

Here are the results on the XNLI test set:

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• Hardware Type: 4x GPUs NVIDIA A100 SXM4 80GB
• Hours used: 7 hours
• Compute Region: France