RoBERTa-NLI-Group71

language: en license: cc-by-4.0 tags:

• text-classification repo: https://huggingface.co/awashh/RoBERTa-NLI-Group71

---

Model Card for j34330vk-q26752aa-NLI

<!-- Provide a quick summary of what the model is/does. -->

This is a Natural Language Inference (NLI) classification model that was trained to detect if a hypothesis is true based on a premise.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This model is based upon a RoBERTa model that was fine-tuned on 26.9K pairs of premise-hypothesis texts.

• Developed by: Awab Alshami and Vansh Kharbanda
• Language(s): English
• Model type: Supervised
• Model architecture: Transformers
• Finetuned from model [optional]: roberta-base

Model Resources

<!-- Provide links where applicable. -->

• Repository: https://huggingface.co/FacebookAI/roberta-base
• Paper or documentation: https://arxiv.org/pdf/1907.11692.pdf

Training Details

Training Data

<!-- This is a short stub of information on the training data that was used, and documentation related to data pre-processing or additional filtering (if applicable). -->

26.9k pairs of premise-hypothesis texts.

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Training Hyperparameters

<!-- This is a summary of the values of hyperparameters used in training the model. -->

  - learning_rate: 2e-05
  - train_batch_size: 32
  - eval_batch_size: 32
  - num_epochs: 8

Speeds, Sizes, Times

<!-- This section provides information about how roughly how long it takes to train the model and the size of the resulting model. -->

  - overall training time: 1.2 hours
  - duration per training epoch: 9 minutes
  - model size: 600 MB

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data & Metrics

Testing Data

<!-- This should describe any evaluation data used (e.g., the development/validation set provided). -->

A subset of the development set provided, amounting to 6.7K pairs.

Metrics

<!-- These are the evaluation metrics being used. -->

  - Precision: 0.882
  - Recall: 0.879
  - F1-score: 0.880
  - Accuracy: 0.880

Results

The model obtained a precision score of 88.2%, a recall score of 87.9%, an F1-score of 88% and an accuracy of 88%.

Technical Specifications

Hardware

  - RAM: at least 22.5 GB
  - Storage: at least 2GB,
  - GPU: A100

Software

  - Transformers 4.18.0
  - Pytorch 1.11.0+cu113

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

Any inputs (concatenation of two sequences) longer than 512 subwords will be truncated by the model.