gliner-small-pii-onnx-int8

gliner-small-pii-onnx-int8

GLiNER PII detection, exported to ONNX and quantized to INT8 so it runs on-device with ONNX Runtime — no Python, no GPU, no network. 188 MiB of weights, CPU inference.

This is a conversion of vicgalle/gliner-small-pii. The weights are not retrained; only the format changed.

What was changed

1. Exported to ONNX with GLiNER.export_to_onnx(quantize=False).
2. Patched the span_mask graph input from BOOL to INT64, inserting a Cast node back to BOOL. Several ONNX Runtime language bindings — including the Objective-C/Swift one — cannot feed a boolean tensor, so the input is named span_mask_int64 in this build.
3. Quantized dynamically to INT8 with onnxruntime.quantization.quantize_dynamic (weight_type=QInt8).

The export script ships with the consuming library, so the conversion recipe is public.

Files

The tarball holds the same four files, flat, gzip with no timestamp. It exists for clients that fetch one archive and verify one hash; the loose files are equivalent.

Pin a commit SHA in download URLs, not main:

https://huggingface.co/blinkwrite-ai/gliner-small-pii-onnx-int8/resolve/<commit-sha>/model.onnx

Graph

Output: logits, [batch, words, max_width, num_labels]. Apply a sigmoid and keep spans above your threshold.

Configuration that must match the weights

• max_width: 12 — spans are enumerated up to 12 words. This is a property of the file, not a tuning knob: change it and span_idx stops lining up with what the model was trained on.
• span_mode: markerV0, max_len: 768, encoder microsoft/deberta-v3-small.

Labels and the score threshold are not fixed by the weights. GLiNER takes its label set at inference time, so any labels work. One caveat: the model returns an index into the array you passed in, so the array used for inference must be the one used to read the result back.

A 17-label set tuned for chat and email prose, at threshold 0.2:

phone number, email address, credit card number, address, social security number, date of birth, bank account number, password, pin code, ip address, dollar amount, passport number, driver license number, tax id, api key, access token, secret key

Limitations

Best-effort NER, not a guarantee. Recall is not 100%, and a PII detector that misses one span has leaked it. Treat the output as a filter, not a compliance control.

• Tuned and evaluated on English chat and email prose. Other languages and other registers — logs, code, forms — are untested.
• The 0.2 threshold trades precision for recall on purpose. Raise it and more PII gets through; lower it and ordinary text gets flagged.
• INT8 quantization costs some accuracy against the fp32 base model. If you need the base model's exact behaviour, use the base model.
• Person names are not in the label set above. GLiNER can detect them if you pass a person label, but name detection in short informal text is the weakest case.

License and attribution

Apache-2.0, inherited from the base model.

• vicgalle/gliner-small-pii — the base model, Apache-2.0.
• gliner-community/gliner_small-v2.5 — which that model fine-tunes, Apache-2.0.
• GLiNER, Zaratiana et al., 2023 — arXiv:2311.08526.
• Encoder: microsoft/deberta-v3-small.

Modification stated per Apache-2.0 §4(b): ONNX export, span_mask dtype patch, and INT8 dynamic quantization. No retraining.