srl-pt_xlmr-large

XLM-R large fine-tuned on Portuguese semantic role labeling

Model description

This model is the xlm-roberta-large fine-tuned on Portuguese semantic role labeling data. This is part of a project from which resulted the following models:

• liaad/srl-pt_bertimbau-base
• liaad/srl-pt_bertimbau-large
• liaad/srl-pt_xlmr-base
• liaad/srl-pt_xlmr-large
• liaad/srl-pt_mbert-base
• liaad/srl-en_xlmr-base
• liaad/srl-en_xlmr-large
• liaad/srl-en_mbert-base
• liaad/srl-enpt_xlmr-base
• liaad/srl-enpt_xlmr-large
• liaad/srl-enpt_mbert-base
• liaad/ud_srl-pt_bertimbau-large
• liaad/ud_srl-pt_xlmr-large
• liaad/ud_srl-enpt_xlmr-large

For more information, please see the accompanying article (See BibTeX entry and citation info below) and the project's github.

Intended uses & limitations

How to use

To use the transformers portion of this model:

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("liaad/srl-pt_xlmr-large")
model = AutoModel.from_pretrained("liaad/srl-pt_xlmr-large")

To use the full SRL model (transformers portion + a decoding layer), refer to the project's github.

Limitations and bias

• This model does not include a Tensorflow version. This is because the "type_vocab_size" in this model was changed (from 1 to 2) and, therefore, it cannot be easily converted to Tensorflow.

Training procedure

The model was trained on the PropBank.Br datasets, using 10-fold Cross-Validation. The 10 resulting models were tested on the folds as well as on a smaller opinion dataset "Buscapé". For more information, please see the accompanying article (See BibTeX entry and citation info below) and the project's github.

Eval results

BibTeX entry and citation info

@misc{oliveira2021transformers,
      title={Transformers and Transfer Learning for Improving Portuguese Semantic Role Labeling}, 
      author={Sofia Oliveira and Daniel Loureiro and Alípio Jorge},
      year={2021},
      eprint={2101.01213},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}