multinerd-multilingual-ner-mdeberta

MultiNERD 15-Domain Multilingual NER (mDeBERTa-v3-base)

This model is a high-accuracy, multi-domain Multilingual Named Entity Recognition (NER) model fine-tuned on the MultiNERD dataset across 10 languages.

Built on top of microsoft/mdeberta-v3-base, it achieves exceptional entity extraction performance across 15 diverse multi-domain categories (31 BIO output logits).

Developed by Rishabh Kumar.

---

1. Model Description & Architecture

• Base Model: microsoft/mdeberta-v3-base (278M parameters, 12 layers, 768 hidden size, 250k multilingual vocabulary)
• Fine-Tuning Method: LoRA (r=32, alpha=32, target modules: query_proj, key_proj, value_proj, intermediate.dense, output.dense) merged into standalone base weights.
• Precision: FP16 (torch.float16, ~530 MB footprint)
• Task: Multilingual Multi-Domain Token Classification / NER
• Languages Supported (10): English (en), German (de), Spanish (es), French (fr), Italian (it), Dutch (nl), Polish (pl), Portuguese (pt), Russian (ru), Chinese (zh).

---

2. Dataset & Preprocessing Pipeline

A. Source Data & Splits

• Source: Babelscape MultiNERD benchmark (Babelscape/multinerd).
• Train Set: 1,339,200 sentences across 10 languages.
• Validation Set: 162,697 sentences.
• Test Set: 167,993 sentences.

B. Preprocessing & Formatting

1. CoNLL Standardization: Converted from Hugging Face dataset structures into 2-column CoNLL format (token tag) with blank lines preserving sentence boundaries.
2. Subword Token Alignment: Aligned with mDeBERTa-v3-base SentencePiece tokenizer with add_prefix_space=True. The first subword token receives the entity BIO label, while sub-tokens receive -100 label masking.
3. Label Preservation: Preserves 15 granular categories into a 31 BIO label vocabulary (id_to_label.json).

---

3. MultiNERD Multi-Domain Taxonomy (15 Classes / 31 BIO Tags)

MultiNERD covers standard NER entities plus 11 specialized domain categories:

---

4. Training Results & Metrics

The model was fine-tuned for 5 epochs (~418.5k steps) using AdamW (lr=2e-5, linear scheduler, effective batch size 16).

Validation Performance (Best Checkpoint - Epoch 5)

Per-Epoch Progression

---

5. How to Use

Quick Inference via Hugging Face Pipeline

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="Rishabh157/multinerd-multilingual-ner-mdeberta",
    aggregation_strategy="simple"
)

text = "The Hubble Space Telescope observed the Andromeda Galaxy."
entities = ner(text)

for entity in entities:
    print(f"{entity['word']:<25} | {entity['entity_group']:<15} | Score: {entity['score']:.4f}")

PyTorch Direct Usage

import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification

model_name = "Rishabh157/multinerd-multilingual-ner-mdeberta"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)

text = "Marie Curie descubrió el polonio y el radio en París."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.argmax(outputs.logits, dim=2)

tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
labels = [model.config.id2label[p.item()] for p in predictions[0]]

for token, label in zip(tokens, labels):
    if label != "O":
        print(f"{token:<20} -> {label}")

---

6. Citation

@misc{kumar2026multinerd,
  author = {Rishabh Kumar},
  title = {MultiNERD 15-Domain Multilingual NER with mDeBERTa-v3},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rishabh157/multinerd-multilingual-ner-mdeberta}}
}