NerGuard-0.3B

NerGuard-0.3B is a multilingual transformer model for Personally Identifiable Information (PII) detection, built on mDeBERTa-v3-base. It performs token-level classification across 20 PII entity types using BIO tagging, covering names, addresses, government IDs, financial data, and contact information across 8 European languages.

Trained on 500K+ samples from AI4Privacy, it achieves F1-macro 99.63% on in-distribution validation. On the out-of-distribution NVIDIA Nemotron-PII benchmark (1,000 samples, 7-system comparison), the base model ranks 4th out of 7 systems on F1-macro and 3rd on Entity-F1 — without any LLM augmentation. For the full hybrid system with entropy-based LLM routing (which ranks 1st on both F1-macro and F1-micro), see the NerGuard GitHub repository.

Note on labels: The model outputs its native AI4Privacy label space (e.g., GIVENNAME, SURNAME, SOCIALNUM). The NerGuard pipeline includes a semantic alignment layer that maps these to benchmark-specific label spaces (e.g., NVIDIA Nemotron-PII uses first_name, ssn).

Supported Entity Types

Evaluation Results

In-Distribution: AI4Privacy (validation split)

Out-of-Distribution: NVIDIA Nemotron-PII (1,000 samples)

Tier 2 evaluation: semantic alignment over 16 comparable entity types. Seven systems compared.

The base model (no LLM) achieves 33 ms median latency. The entropy-gated hybrid adds +8.94 pt F1-macro by routing only uncertain spans (~3% of tokens) to an LLM for disambiguation.

Usage

from transformers import pipeline

ner = pipeline(
    "token-classification",
    model="exdsgift/NerGuard-0.3B",
    aggregation_strategy="simple"
)

results = ner("My name is John Smith and my email is [email protected]")
for entity in results:
    print(f"{entity['word']} -> {entity['entity_group']} ({entity['score']:.2%})")
# John  -> GIVENNAME (99.82%)
# Smith -> SURNAME   (99.71%)
# [email protected] -> EMAIL (99.54%)

For the full hybrid pipeline with LLM routing and regex validation:

from src.inference.tester import PIITester

tester = PIITester(model_path="exdsgift/NerGuard-0.3B")
entities = tester.get_entities("John Smith, SSN: 078-05-1120, email: [email protected]")

Training Details

Citation

@mastersthesis{durante2026nerguard,
  title     = {Engineering a Scalable Multilingual PII Detection System
               with mDeBERTa-v3 and LLM-Based Validation},
  author    = {Durante, Gabriele},
  year      = {2026},
  school    = {University of Verona},
  department = {Department of Computer Science}
}