electra_large_discriminator_squad2_custom_dataset

Question & Answering Model for 'Save Your Minutes' from Dobby-AI

Electra_Large Discriminator fine-tuned on SQuAD2.0 and custom QA dataset

This model is ahotrod/electra_large_discriminator_squad2_512 trained on additional custom dataset as:

!python3 run_squad.py  --model_type electra \
						--model_name_or_path /content/electra_large_512 \
						--do_lower_case \
						--output_dir /content/model/\
						--do_train \
						--train_file $data_dir/additional_qa.json\
						--version_2_with_negative \
						--do_lower_case \
						--num_train_epochs 3 \
						--weight_decay 0.01 \
						--learning_rate 3e-5 \
						--max_grad_norm 0.5 \
						--adam_epsilon 1e-6 \
						--max_seq_length 512 \
						--doc_stride 128 \
						--threads 12 \
						--logging_steps 50 \
						--save_steps 1000 \
						--overwrite_output_dir \
						--per_gpu_train_batch_size 4

We used Google Colab for training the model,