bcms-bertic-comtext-sr-legal-ner-ekavica

BERTić-COMtext-SR-legal-NER-ekavica

BERTić-COMtext-SR-legal-NER-ekavica is a variant of the BERTić model, fine-tuned on the task of named entity recognition in Serbian legal texts written in the Ekavian pronunciation. The model was fine-tuned for 20 epochs on the Ekavian variant of the COMtext.SR.legal dataset.

Benchmarking

This model was evaluated on the task of named entity recognition in Serbian legal texts. The model uses a newly developed named entity schema consisting of 21 entity types, tailored for the domain of Serbian legal texts, and encoded according the the IOB2 standard. The full entity list is available on the COMtext.SR GitHub repository.

This model was compared with SrBERTa, a model specially trained on Serbian legal texts, fine-tuned for 20 epochs for named entity recognition using the Ekavian variant of the COMtext.SR.legal corpus of legal texts. Token-level accuracy and F1 (macro-averaged and per-class) were used as evaluation metrics and gold tokenized text was taken as input.

Two evaluation settings for both models were considered:

• Default - only the entity type portion of the NE tag is considered, effectively ignoring the "B-" and "I-" prefixes
• Strict - the entire NE tag is considered

For the strict setting, per-class results are given separately for each B-CLASS and I-CLASS tag. In addition, macro-averaged F1 scores are presented in two variants - one where the O (outside) class is ignored, and another where it is treated equally to other named entity classes.

BERTić-COMtext-SR-legal-NER-ekavica and SrBERTa were fine-tuned and evaluated on the COMtext.SR.legal.ekavica corpus using 10-fold CV.

The code and data to run these experiments is available on the COMtext.SR GitHub repository.

Results