pii-master-ner-m

pii-master-ner-m 0.3.1

Token tagger for PII / PHI detection, distilled from kalyan-ks/ettin-68m-nemotron-pii on nvidia/Nemotron-PII. Runs on one CPU core.

• dilated depthwise-separable CNN tagger, d=128 x 6 layers, 6.57M parameters
• 111 BIO classes over 55 Nemotron entity types, crosswalked to 25 HIPAA-mapped types
• onnx-fp32, 30.0 MB
• Confidence calibration: isotonic, per entity type
• Source commit: a02891b99f59e83e76b7e65c4485b121e5e6d720
• Trained on: nvidia/Nemotron-PII

Intended use

Input is plain text; output is character spans typed against the 18 HIPAA Safe Harbor identifier categories. It is designed to run behind the Stage 1 rules tier, which supplies checksum-validated spans it cannot beat, and which suppresses failure classes it re-introduces on its own (see Limitations).

Not a de-identification guarantee. It is a detector that helps a reviewer. Safe Harbor de-identification is a legal determination this model cannot make.

How good is it?

At the level you act on: does this document contain PII?

Measured on 3,000 held-out Nemotron documents; a document counts as sensitive if it carries a gold span of a type we model. False alarms are measured on 14 adversarial negatives -- order numbers, chart numbers, subscriber ids -- not on easy ones.

At the span level: is the tag right?

Exact (type, start, end) match, fused with the rules tier. F2 weights recall four times as heavily as precision, which is closer to this system's cost matrix than F1; both are shown.

<details><summary>Per type</summary> </details>

Limitations that matter

• These scores are for Nemotron-PII, and they do NOT generalise. Measured on ai4privacy/pii-masking-300k -- a different corpus, label space, document style and locale -- in-scope strict span recall is 0.385 against 0.914 here, and document-level recall 0.870 against 0.998. Format-anchored types transfer intact (EMAIL 0.943, IP 0.988); learned semantic types collapse (names and addresses ~0.30, mostly boundary errors on structured JSON text). Deep mode still roughly doubles the rules on that corpus, so the cascade earns its place -- but budget for the lower number on text unlike the training set.
• Synthetic training data. Nemotron-PII is generated, not real, and the standard clinical benchmark (n2c2/i2b2 2014) requires a data use agreement and was not used. No number here describes real clinical text.
• The demographic slice is synthetic too. Name recall varies by only 0.020 across race/ethnicity groups, which sounds excellent and mostly reflects names drawn from a generator rather than from the world. It is a real gate that would catch a large disparity; it certifies synthetic names only.
• The PII-vs-PHI split has no external gold. Nemotron has no document labels and no medical-context annotation, so that boundary is only scored on a 39-document authored corpus. It is the weakest link in the evaluation.
• Credit card numbers are deliberately suppressed. 88% of the training corpus's cards fail the Luhn checksum, so the serving path re-validates and drops them. Measured F1 on that type is ~0.18 against this corpus, and that is correct behaviour rather than a defect.
• US / English scope. Non-US identifier formats are out of scope.
• Calibrated for THIS model on THIS corpus. A threshold tuned here is not automatically right for another text distribution.
• It can be confidently wrong. Adversarial near-misses are its documented failure class, which is why the serving path ships a confidence floor, checksum re-validation and rule fusion. Do not run it bare.

Licensing

The code is MIT (see LICENSE). The model is a derivative work of its training data, and inherits obligations from every corpus below.

• nvidia/Nemotron-PII — CC BY 4.0; attribution to NVIDIA is required when redistributing this model or its outputs.
• teacher kalyan-ks/ettin-68m-nemotron-pii — MIT.

Verify before you trust it

python training/package.py verify <this directory>

The weights ship in model.onnx.data beside model.onnx. A package whose weights are corrupted without changing their size still loads and still answers: on the real artifact a flipped kilobyte turned an MRN into a USER_ID at 0.87 confidence, which is a silent PHI miss. Only the checksum catches that.