roberta-base-bne-squad-2.0-es

This is the BSC-TeMU/roberta-base-bne model (source) trained on the squad_es v2.0.0 dataset (source).

Current achievement: em=58.80, f1=67.40

Results:

{
    "epoch": 4.0,
    "eval_HasAns_exact": 48.51551956815115,
    "eval_HasAns_f1": 65.70745010262016,
    "eval_HasAns_total": 5928,
    "eval_NoAns_exact": 69.0893760539629,
    "eval_NoAns_f1": 69.0893760539629,
    "eval_NoAns_total": 5930,
    "eval_best_exact": 58.804182830156854,
    "eval_best_exact_thresh": 0.0,
    "eval_best_f1": 67.39869828034618,
    "eval_best_f1_thresh": 0.0,
    "eval_exact": 58.804182830156854,
    "eval_f1": 67.39869828034568,
    "eval_samples": 12211,
    "eval_total": 11858
}

Training script:

python -m torch.distributed.launch --nproc_per_node=3 ./run_qa.py \
    --model_name_or_path BSC-TeMU/roberta-base-bne \
    --dataset_name squad_es \
    --dataset_config_name v2.0.0 \
    --do_train \
    --do_eval \
    --learning_rate 3e-5 \
    --num_train_epochs 4 \
    --max_seq_length 384 \
    --doc_stride 128 \
    --output_dir ./models/roberta-base-bne-squad-2.0-es/ \
    --per_device_eval_batch_size=8   \
    --per_device_train_batch_size=8   \
    --version_2_with_negative \
    --ddp_find_unused_parameters=False \
    --overwrite_output_dir \