roberta-large-bne-squad-2.0-es

This is the BSC-TeMU/roberta-large-bne model (source) trained on the squad_es v2.0.0 dataset (source).

Current achievement: em=60.21, f1=68.61

Results:

{
    "epoch": 4.0,
    "eval_HasAns_exact": 48.44804318488529,
    "eval_HasAns_f1": 65.24520506718169,
    "eval_HasAns_total": 5928,
    "eval_NoAns_exact": 71.97301854974705,
    "eval_NoAns_f1": 71.97301854974705,
    "eval_NoAns_total": 5930,
    "eval_best_exact": 60.22094788328555,
    "eval_best_exact_thresh": 0.0,
    "eval_best_f1": 68.6181122987237,
    "eval_best_f1_thresh": 0.0,
    "eval_exact": 60.2125147579693,
    "eval_f1": 68.60967917340695,
    "eval_samples": 12203,
    "eval_total": 11858
}

Training script:

python -m torch.distributed.launch --nproc_per_node=3 ./run_qa.py \
    --model_name_or_path BSC-TeMU/roberta-large-bne \
    --dataset_name squad_es \
    --dataset_config_name v2.0.0 \
    --do_train \
    --do_eval \
    --learning_rate 3e-5 \
    --num_train_epochs 4 \
    --max_seq_length 384 \
    --doc_stride 128 \
    --output_dir ./models/roberta-large-bne-finetuned-squad-es/ \
    --per_device_eval_batch_size=24   \
    --per_device_train_batch_size=12   \
    --version_2_with_negative \
    --ddp_find_unused_parameters=False \