FYP_qa_final

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

FYP_qa_final

This model is a fine-tuned version of deepset/deberta-v3-base-squad2 on an MRQA dataset. It achieves the following results on the evaluation set:

• Loss: 2.7493

Model description

This model is trained for performing extractive question-answering tasks for academic essays.

Intended uses & limitations

More information needed

Training and evaluation data

The dataset used for training is listed below according to training sequences:

1. MRQA(train split)
2. UCLNLP/adversarial_qa
3. mbartolo/synQA
4. MRQA(test split)*This model

Training procedure

The training approach uses the fine-tuning approach of transfer learning on the pre-trained model to perform NLP QA tasks. Each time a model was trained with one dataset only and saved as the PTMs for the next training. This model is the last model that trained with MRQA(test split).

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3
• mixed_precision_training: Native AMP

Training results

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2