MiniCPM5-1B-PII-tagger-lora-100k

MiniCPM5-1B PII Tagger (LoRA, full 100K)

A LoRA adapter on openbmb/MiniCPM5-1B that turns the model into a token-classification PII/PHI detector, trained on the full 100K-example train split of nvidia/Nemotron-PII. A 20K-pilot version of this adapter preceded this run (now archived) — this full run scores higher and is the one to use.

What it does

Given English text, the model predicts one of 55 PII/PHI category labels per token (IO scheme): first_name, last_name, date_of_birth, ssn, street_address, email, phone_number, credit_debit_card, blood_type, medical_record_number, api_key, password, ipv4, mac_address, and 40 more.

Label list (sorted): account_number, age, api_key, bank_routing_number, biometric_identifier, blood_type, certificate_license_number, city, company_name, coordinate, country, county, credit_debit_card, customer_id, cvv, date, date_of_birth, date_time, device_identifier, education_level, email, employee_id, employment_status, fax_number, first_name, gender, health_plan_beneficiary_number, http_cookie, ipv4, ipv6, language, last_name, license_plate, mac_address, medical_record_number, national_id, occupation, password, phone_number, pin, political_view, postcode, race_ethnicity, religious_belief, sexuality, ssn, state, street_address, swift_bic, tax_id, time, unique_id, url, user_name, vehicle_identifier.

Results

Evaluated on 2,000 held-out examples from the nvidia/Nemotron-PII test split (seed 42 shuffle). Metrics are exact token/span match against char-offset-derived gold labels; a span is a maximal run of consecutive tokens with the same non-O label.

The full 100K split is 5× the data and lifted span F1 by +2.4 points over the pilot. Label vocabulary is identical in both runs (55 categories), so the adapters are interchangeable with respect to label space.

Usage

import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification
from peft import PeftModel

labels = ["account_number", "age", "api_key", "bank_routing_number", "biometric_identifier",
"blood_type", "certificate_license_number", "city", "company_name", "coordinate", "country",
"county", "credit_debit_card", "customer_id", "cvv", "date", "date_of_birth", "date_time",
"device_identifier", "education_level", "email", "employee_id", "employment_status", "fax_number",
"first_name", "gender", "health_plan_beneficiary_number", "http_cookie", "ipv4", "ipv6",
"language", "last_name", "license_plate", "mac_address", "medical_record_number", "national_id",
"occupation", "password", "phone_number", "pin", "political_view", "postcode", "race_ethnicity",
"religious_belief", "sexuality", "ssn", "state", "street_address", "swift_bic", "tax_id", "time",
"unique_id", "url", "user_name", "vehicle_identifier"]
id2label = {i: l for i, l in enumerate(labels)}
label2id = {l: i for i, l in enumerate(labels)}

base = AutoModelForTokenClassification.from_pretrained(
    "openbmb/MiniCPM5-1B", id2label=id2label, label2id={l: i for i, l in enumerate(labels)},
    dtype="bfloat16",
)
model = PeftModel.from_pretrained(base, "luispoveda93/MiniCPM5-1B-PII-tagger-lora-100k")
tok = AutoTokenizer.from_pretrained("luispoveda93/MiniCPM5-1B-PII-tagger-lora-100k")

text = "My date of birth is 1987-05-22 and I live at 87 Avenida De La Estrella."
enc = tok(text, return_tensors="pt", return_offsets_mapping=True, truncation=True, max_length=1024)
offsets = enc.pop("offset_mapping")[0].tolist()
with torch.no_grad():
    preds = model(**enc).logits.argmax(-1)[0].tolist()
for (s, e), p in zip(offsets, preds):
    if p != 0 and not (s == 0 and e == 0):
        print(text[s:e], "->", id2label[p])

Adjacent predicted tokens with the same label form one entity span.

Training details

• Base: openbmb/MiniCPM5-1B (LlamaForTokenClassification head, 55 labels, bf16, SDPA attention)
• Data: all 100,000 examples of nvidia/Nemotron-PII default/train after a seed-42 shuffle; spans mapped to tokens via fast-tokenizer offset_mapping (IO scheme, -100 on special tokens); max_length=1024 (only ~0.07% of spans lost to truncation, measured on the pilot sample)
• LoRA: r=16, alpha=32, dropout=0.1, target modules q/k/v/o/gate/up/down_proj — 11.3M trainable params (1.27%)
• Hyperparameters: lr 2e-4 (cosine, no warmup), effective batch 32 (8 × grad-accum 4), 1 epoch (3,125 optimizer steps), bf16, seed 42
• Hardware: 1× A10G (24 GB), peak GPU memory 22.39 GB, train runtime 2 h 41 min

Known data caveats (from nvidia/Nemotron-PII)

• ~1.5% of spans have a char-level mismatch between spans[i].text and text[start:end] in the source dataset (measured over 182K spans in the pilot sample). Offsets were treated as ground truth for labeling.
• 55 unique labels confirmed across the full 100K + 2K sample (842,660 spans, ~4.2 spans/example).

Limitations

• Single epoch, no per-label breakdown — rare categories likely underperform.
• Synthetic, persona-grounded text only; real-world documents may shift.
• Token-level boundaries are exact-match; adjacent same-label spans merge into one prediction.

Job provenance

Trained with HF Jobs (pii-tagger-full-100k-minicpm5-a10g, job 6aa0708832d5d0c22c5ae6f2), A10G-small, 2026-09-08. Pilot: pii-tagger-pilot-20k-minicpm5-a10g-v2-push (job 6aa0653d32d5d0c22c5ae52c).