span-detector-ro

Span detector (Romanian) — entity detection for end-to-end RE

Dragoș Mitruț Vasile · Elena-Simona Apostol · Stefan-Adrian Toma · Adrian Paschke · Ciprian-Octavian Truică

A token-classification model (FacebookAI/xlm-roberta-large, 560M) that tags the two entity spans e1 and e2 in a plain Romanian sentence, using a five-label BIO scheme (O, B-E1, I-E1, B-E2, I-E2). It is the first stage of the end-to-end pipeline baseline: its predicted spans are wrapped in entity markers and passed to the relation classifier (DS4AI-UPB/xlmr-large-ro-re).

The SemEval-2010 Task 8 entities are common nominals, not named entities, so an off-the-shelf NER model does not transfer — this detector is trained on the task's own spans instead.

Results (span F1-Score, held-out validation split)

Training

• Base: FacebookAI/xlm-roberta-large
• Token classification, 5 BIO labels
• 5 epochs, batch size 16, LR 2e-5, 10% warmup, weight decay 0.01
• Best checkpoint by span F1-Score on a 10% validation split
• Single NVIDIA A100 40GB, about 6 min

Usage

from transformers import AutoModelForTokenClassification, AutoTokenizer

tok = AutoTokenizer.from_pretrained("DS4AI-UPB/span-detector-ro")
model = AutoModelForTokenClassification.from_pretrained("DS4AI-UPB/span-detector-ro")

The full pipeline (span detection then relation classification) is in pipeline_e2e.py in the code repository.

Citation

@misc{vasile2026crosslingual,
  title  = {Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian},
  author = {Vasile, Drago\c{s}-Mitru\c{t} and Apostol, Elena-Simona and Toma, \c{S}tefan-Adrian and Paschke, Adrian and Truic\u{a}, Ciprian-Octavian},
  year   = {2026},
  note   = {Preprint}
}