blackbar-nano

blackbar-nano

Local-first PII detection and redaction. A compact NER model that finds 35 types of personally identifiable information in English text. No GPU needed, about 38 ms per chat-sized message on a base Apple M1 (8GB), no remote inference API calls, no data leaving the machine.

Try the browser demo. The model downloads the first time you use it, then runs on your device. The text you paste is not uploaded.

• Architecture: ModernBERT-base encoder + span detector + fixed 35-way classification head (149M-parameter encoder, 151.4M parameters total, fp32)
• Training: 570,713 effective rows per epoch, combining 476,441 rows converted from public annotated datasets with 47,136 unique conversational rows generated with Anthropic Claude (upsampled 2x). Full source breakdown: TRAINING_DATA.md. Training data is not released.
• License: Apache-2.0 (weights + inference code)
• By: safetype

Usage

pip install blackbar-nano

from blackbar_nano import Redactor

redactor = Redactor()
print(redactor.redact("Call Sarah at 503-555-0142"))
# Call [FIRST_NAME] at [PHONE_NUMBER]

The custom span-classification head here is loaded through the blackbar-nano package, not through a standard Transformers token-classification pipeline, the checkpoint on this page won't load with AutoModelForTokenClassification. Source and evaluation scripts: GitHub.

ONNX (browser / onnxruntime)

The onnx/ folder has the same v1 weights exported for ONNX Runtime. It's split into two graphs:

Why two graphs? In browsers (onnxruntime-web 1.27.0) the WebGPU execution provider gets the head's per-span gather wrong. The encoder is fine. So you run the encoder on WebGPU and the head on WASM/CPU. You get GPU speed and the head math stays exact.

The two graphs compose fine in any onnxruntime. Feed the encoder's hidden output into the head, along with the span index arrays. Tokenization, span-candidate generation and thresholding all live outside the graphs. See the reference implementation in the blackbar-nano package.

Fidelity: on the released 3,895-row eval, this fp16 encoder plus fp32 head scores the same as the PyTorch checkpoint (96.9% coverage, 10.9% false-alarm share). Exported with torch dynamo at opset 18. The fp16 conversion keeps the span-pooling ops in fp32.

Encoder chunks (only if you're short on memory)

Skip this if you're not. The single encoder file is simpler.

Here's the problem. onnxruntime-web needs about twice a model's file size in scratch space while it builds a session. On iOS Safari the 300 MB encoder goes past what a tab is allowed to use, and the browser kills the page. You don't get an error you can catch. The page just reloads.

The chunk files are the same weights cut into four graphs. You load one, build it, drop it, then move to the next. Peak memory then follows the biggest chunk (about 75 MB) instead of the whole encoder. Nothing is quantized. Nothing is retrained. The total size is the same.

To use them, run the chunks in order. Each one lists what it needs in its inputs, and those all come from input_ids, attention_mask, or an earlier chunk. The last chunk gives you hidden. The wiring is in blackbar_nano_v1_encoder_fp16_chunks.json.

One thing to know. Chained chunks are not bit-identical to the single file. fp16 values round at each graph boundary, so you get about one fp16 ULP of drift starting at the third chunk. On a 400-row eval slice this changed no predictions. Same coverage, same misses, same false-alarm share.

sha256:

01ef023d13da84380ca9e5001f4774f7601dd1563f9f339b8cbc0f4855a70560  blackbar_nano_v1_encoder_fp16.onnx
7445e42b0e76717447da1470c8d3fb4889f4198ffea7523eb11e987ca9fb7045  blackbar_nano_v1_head_fp32.onnx
5622ec96b57374083a8fcf92c117ce8b7d282ab6503e4d9a902ec0c0be462f0f  blackbar_nano_v1_encoder_fp16_chunk0.onnx
6349a704cfb1e3478a55eca719cf6a58af6a5b9d3a68007e6ed2b319a2d8e76a  blackbar_nano_v1_encoder_fp16_chunk1.onnx
224031bd262e8ffdd7c57180086983a7016b8261101205d2434315fc946fa980  blackbar_nano_v1_encoder_fp16_chunk2.onnx
3c3f425557493d8ef7d4a06638d6eda5b0c533190ee7693ec8931ee825590706  blackbar_nano_v1_encoder_fp16_chunk3.onnx

Labels (35)

first_name last_name full_name person date_of_birth email phone_number address street_address city state_or_region postal_code country national_id_number passport_number drivers_license_number license_number tax_id account_number routing_number iban card_number card_cvv username ip_address account_id password api_key sensitive_date medical_record_number health_insurance_id medical_condition medication case_number url

Benchmarks

All head-to-head comparisons run both models on byte-identical text. The paper-reported SPY figure below is shown separately and used a different generated instance.

These are task-level comparisons of the released configurations, not architecture-matched experiments. blackbar-nano used its shipped 0.60 entity and 0.50 label thresholds; GLiNER2-PII used its documented 0.50 inference threshold and was not separately threshold-tuned for these comparisons. Scores are restricted to mutually mapped PII concepts where applicable.

Redaction coverage (3,895-row held-out eval, 5,244 PII spans; "coverage" = share of gold PII spans overlapped by at least one prediction. This is the metric that matters when the goal is that secrets end up behind a black bar):

SPY public benchmark (mks-logic/SPY, CC-BY-4.0: legal-forum + medical-consultation documents; exact-match span+type F1 over 7 PII types, label-mapping methodology as in the GLiNER2-PII paper). SPY generates different fake values on every load, so I froze one fixed-seed instance and ran both models on it:

The frozen instance ships with the GitHub repo (eval/run_spy_eval.py), so this is closely reproducible, not just a claim.

Latency (CPU, single message):

The Windows machine is an older mobile workstation chip, not a modern desktop or server CPU, which likely explains most of that gap, though two machines isn't enough to fully separate hardware age from other differences between them. Coverage and stray share matched almost exactly across both machines. Only speed differed, see the verification table below.

A 400-row verification slice ships with the GitHub repo (eval/run_eval.py) as a smaller public check of coverage. It should produce about 97.5%, separate from the 3,895-row evaluation behind the 96.9% headline above. Stray share on that script reads higher, around 15%, than the 10.9% figure above, since the two numbers come from different annotation scopes: the public file only carries gold labels for nano's 35 types, so a prediction landing on some other real entity outside that set counts as a false alarm there. The 10.9% figure and the table below were measured separately, against a broader annotation scope, same 400 rows, same machine, an Apple M1:

That 545 is a different, narrower count than the 594 annotations in the public eval slice. This comparison used a different label filter.

Intended use & how it works

Feed text up to 256 tokens per window (the runtime chunks longer inputs); the model proposes candidate spans, scores each for "is this an entity" (threshold 0.60) and classifies it over the 35 labels (threshold 0.50). Thresholds were selected for redaction recall on held-out data; raise them for higher precision.

Intended for: redacting chat logs, tickets, transcripts, and documents before they leave a trust boundary (e.g. before sending text to a cloud LLM).

Design choices

• ModernBERT as the encoder. It's an encoder-only, bidirectional model with modern architecture improvements and 8,192-token support, pretrained on 2 trillion tokens of English and code. It cleared the quality ceiling the earlier MiniLM-based versions were hitting, while still meeting the CPU-latency goal, measured independently on this model's own CPU benchmark, not inherited from ModernBERT's published numbers, which are mostly reported on GPU.

• Whole-span scoring, not token-level BIO tagging. A common NER design tags each token and reassembles spans from the tags afterward. This model instead enumerates candidate spans directly and scores each whole span with the entity and label heads, which matches how PII actually appears in text, as chunks, not something built up token by token.

• Candidates built from whitespace-delimited units, capped at 12. "Word" here technically means any non-whitespace run, with punctuation-trimmed variants also generated, then mapped onto tokenizer boundaries. Every gold span in the public 400-row eval file fits comfortably within that cap, the longest is 9 units. Raising the cap would increase the number of candidates the model has to score roughly in proportion to the cap itself, without the public evaluation data showing any need for it.

• A 256-token operating window, with overlapping chunks for longer text. A short message doesn't get padded out to that length, so raising the window wouldn't slow down the common short case. The real reason is that 256 matches what the model was actually trained and evaluated on, and it bounds the worst-case compute, memory, and behavior for a long document. Chunks run at roughly a 252-token budget with a 30-unit overlap, so a span sitting right on a chunk boundary still gets a complete chance to be scored, though that's not a guarantee of full detection coverage across a long document.

• Two independent acceptance gates, then a combined ranking score. A span has to clear an entity-confidence gate and a separate label-confidence gate before it's kept, and once both pass, the two scores get multiplied together into one combined score used to resolve overlapping candidates. The two gates aren't directly comparable numbers; they come from different heads with different score distributions, but keeping them independent lets entity-capture and label-certainty each get calibrated on their own.

• Low-confidence spans get dropped, not tagged with a generic fallback. A generic fallback isn't impossible for a fixed classifier. In principle a 36th "unknown PII" class could be trained in, but that changes training, calibration, and what the other 35 labels need to learn to separate from. That path wasn't taken here. The real tradeoff stands either way: some borderline spans go uncaught instead of being caught with a generic label, which is part of why the threshold tuning matters.

• Several role-specific name labels folded into one "person" label. Names annotated as lawyer, judge, witness, and a few other roles were all merged, not the role words themselves, but the person names carrying those annotations. For redaction, the role doesn't change what needs to happen. The name gets covered either way. The merge added roughly 19,000 name-span examples, meant to strengthen supervision for the model's historically weakest spot, name detection, though that's the intent behind the decision, not a claim backed by a controlled before and after comparison.

• Public annotated datasets plus targeted synthetic conversational data. Existing public datasets provided broad coverage across PII, legal, biomedical, social, and other annotated text. LLM generation was used to add conversational phrasing, international formats, adversarial near-misses, and other cases that were sparse in the public data. This avoided collecting private user PII specifically for this project, but it does not mean every training example was synthetic: the public corpus includes real-world contracts, abstracts, tweets, and other text. Full breakdown: TRAINING_DATA.md.

• Mixed-precision training, with fp32 forced for the safety-critical math, and a fully fp32 released model. Training used autocast for speed, with bf16 or fp16 where bf16 was unsupported, but the span-pooling math, the span network, and the classification heads were explicitly forced back to fp32. That wasn't one bug, it was two separate ones: reduced-precision math in the pooling step's cumulative sum caused one NaN failure, and a completely different issue, the default attention path misbehaving on padded batches, caused another. Both got fixed. The tested fp32 CPU runtime already met the latency target, so no separate reduced-precision inference path was released.

Limitations (honest list)

• URLs are the weak label. The detector misses roughly two-thirds of URLs (SPY URL F1 0.11–0.30). URLs are also the most regex-able PII type. Pair the model with a one-line URL regex pre-pass as a workaround.
• English only. Not trained or evaluated on other languages.
• Exact boundaries are approximate on multi-word names/addresses. The model may split "John Smith" into two spans. For redaction this is harmless (both halves are covered); for strict span extraction it costs exact-match F1.
• Fixed label set. No custom or user-defined entity types. The head is a closed 35-way classifier.
• Mixed training distribution. Training combined public annotated corpora with synthetic conversational examples. This provides broader register coverage than synthetic data alone, but unusual formats and deployment-specific language may still be underrepresented.
• Only tested on two machines so far, an Apple M1 and one Windows laptop. If you test on other hardware, especially Linux, open an issue with your numbers.

Training data statement

Training used 476,441 converted rows from public annotated datasets and 47,136 unique conversational rows generated with Anthropic Claude. The conversational rows were upsampled 2x, producing an effective training stream of 570,713 rows per epoch. Some source datasets are themselves synthetic, while others contain public real-world text. The project did not collect private user data specifically for training. Full source list, per-dataset licenses, and row counts: TRAINING_DATA.md. The training data itself, and the data-preparation pipelines that built it, are not released.

Not affiliated with the R package bnosac/blackbar or the PyPI desktop utility blackbar.