bert-base-gl-ner

Named Entity Recognition BERT Model for Galician

This repository includes a fine-tuned model for Named Entity Recognition (NER) annotation for Galician. It is a fine-tuned version of marcosgg/bert-base-gl-cased trained for token classification with the BIO tagging scheme.

The model recognises four entity types:

Model description

• Architecture: BertForTokenClassification (BERT base, 12 layers, hidden size 768, 12 attention heads)
• Base model: marcosgg/bert-base-gl-cased
• Language: Galician (gl)
• Task: Named Entity Recognition (token classification)
• Labels (BIO): B-PER, I-PER, B-LOC, I-LOC, B-ORG, I-ORG, B-MISC, I-MISC, O
• Cased: yes
• Maximum sequence length: 512 tokens

Intended use

The model is intended for automatic Named Entity Recognition in Galician text, tagging tokens as person (PER), location (LOC), organisation (ORG) or miscellaneous (MISC) entities.

Training and evaluation data

The training, development and test data are available at:

https://github.com/albinasarymsakova/Named-Entity-Recognition-Resources-for-Galician

The data is distributed in JSON Lines format, with parallel lists of tokens and BIO labels per sentence.

Evaluation results

Test sets

Results (%) on the five individual Galician NER test sets, using the BIO scheme and entity-level (seqeval) scoring:

How to use

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_id = "asarymsakova/bert-base-gl-ner"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

ner = pipeline(
    "token-classification",
    model=model,
    tokenizer=tokenizer,
    aggregation_strategy="simple",
)

text = "A Universidade de Santiago de Compostela está situada en Galicia."
for entity in ner(text):
    print(entity["entity_group"], entity["word"], round(entity["score"], 3))

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 64
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Framework versions

• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1

License

This resource is made available under the terms of the GNU Affero General Public License v3.0 (AGPL-3.0). See https://www.gnu.org/licenses/agpl-3.0.html for the full license text. The resource is distributed without any warranty.

Information and contact

• Albina Sarymsakova — [email protected]
• Marcos Garcia — [email protected]

Citation

If you use this model, please cite this repository and the accompanying NER resources for Galician: https://github.com/albinasarymsakova/Named-Entity-Recognition-Resources-for-Galician