coref-ua

Coreference resolution model for the Ukrainian language

<!-- Provide a quick summary of what the model is/does. -->

The coreference resolution model for the Ukrainian language was trained on the silver Ukrainian coreference dataset using the F-Coref library. The model was trained on top of the XML-Roberta-base model.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

• Developed by: Artem Kramov, Andrii Kursin ([email protected]).
• Languages: Ukrainian
• Finetuned from model: XML-Roberta-base

Model Sources

<!-- Provide the basic links for the model. -->

• Repository: https://github.com/artemkramov/fastcoref-ua/blob/main/README.md
• Demo: Google Colab, Streamlit

Out-of-Scope Use

According to the metrics retrieved from the evaluation dataset, the model is more precision-oriented. Also, there is a high level of granularity of mentions. E.g., the mention "Головний виконавчий директор Андрій Сидоренко" can be divided into the following coreferent groups: ["Головний виконавчий директор Андрій Сидоренко", "Головний виконавчий директор", "Андрій Сидоренко"]. Such a feature can also be used to extract some positions, roles, or other features of entities in the text.

How to Get Started with the Model

Use the code below to get started with the model.

from fastcoref import FCoref
import spacy

nlp = spacy.load('uk_core_news_md')

model_path = "artemkramov/coref-ua"
model = FCoref(model_name_or_path=model_path, device='cuda:0', nlp=nlp)

preds = model.predict(
   texts=["""Мій друг дав мені свою машину та ключі до неї; крім того, він дав мені його книгу. Я з радістю її читаю."""]
)

preds[0].get_clusters(as_strings=False)
> [[(0, 3), (13, 17), (66, 70), (83, 84)],
 [(0, 8), (18, 22), (58, 61), (71, 75)],
 [(18, 29), (42, 45)],
 [(71, 81), (95, 97)]]

preds[0].get_clusters()
> [['Мій', 'мені', 'мені', 'Я'], ['Мій друг', 'свою', 'він', 'його'], ['свою машину', 'неї'], ['його книгу', 'її']]

preds[0].get_logit(
   span_i=(13, 17), span_j=(42, 45)
)

> -6.867196

Training Details

Training Data

The model was trained on the silver coreference resolution dataset: https://huggingface.co/datasets/artemkramov/coreference-dataset-ua.

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Metrics

Two types of metrics were considered: mention-based and the coreference resolution metrics themselves.

Mention-based metrics:

• mention precision
• mention recall
• mention F1

Coreference resolution metrics were calculated as the average values across the following metrics: MUC, BCubed, CEAFE:

• coreference precision
• coreference recall
• coreference F1

Results

The metrics for the validation dataset:

Model Card Authors

Artem Kramov (https://www.linkedin.com/in/artem-kramov-0b3731100/), Andrii Kursin ([email protected])