Somethings

Main Script: QuestionAnswering.py

The script uses HuggingFace library for managing the datasets, importing/exporting models and training the models.

There are various variables at the start of the script.

• train: Training a new model

• PEFT: Whether to use PEFT during training

• tf32/fp16: Mixed precision training choice

• trained_model: Name of trained model (to be pushed to HF Hub)

• train_checkpoint: Checkpoint of training (None by default)

• squad_shift: Whether to include extra data (squadshift)

• base_tokenizer: Tokenizer of base model

• base_model: Pre-trained model

• test: Testing a model

• tokenizer_list/model_list/question_list: Which tokenizer, model and questions to be tested.

CUDA is enabled if applicable. Require user to login into HuggingFace Hub (via command line token or through script) if training. Alternative is to not push to hub, a local repository will be created.

Huggingface repositories created (models created)

• botcon/XLNET_squad_finetuned_large
• botcon/XLNET_squadshift_finetuned_large
• botcon/LUKE_squad_finetuned_large
• botcon/LUKE_squadshift_finetuned_large
• botcon/LUKE_squad_what