openmed-privacy-filter-multilingual-v2-onnx

OpenMed Privacy Filter Multilingual v2 - ONNX

This is an ONNX export of OpenMed/privacy-filter-multilingual-v2, converted for CPU inference with onnxruntime's CPUExecutionProvider. No fine-tuning was performed - this repo only changes the runtime format, not the model's weights or behavior.

About the base model

All credit for the model itself goes to OpenMed. From their model card: privacy-filter-multilingual-v2 performs fine-grained PII extraction across 54 entity categories in 16 languages, built on top of OpenAI's Privacy Filter (a Mixture-of-Experts transformer, 128 experts with top-4 routing per token) with a BIOES token-classification head, and extended with multilingual coverage and expanded entity types by OpenMed and the AI4Privacy community. See the original model card for the full entity list, training details, and citation.

This conversion exists only because no ONNX export was published alongside the original safetensors weights.

What was done for this conversion

The checkpoint's config.json declares a custom model_type/architecture name (openai_privacy_filter / OpenAIPrivacyFilterForTokenClassification). Depending on your installed transformers version this may already be a recognized, first-class architecture (loadable directly via AutoModelForTokenClassification/AutoTokenizer) - that's what this export uses. If you're on an older transformers release without native support, do not substitute transformers' generic gpt_oss classes as a stand-in: despite matching tensor shapes (the checkpoint loads with zero missing/unexpected weight keys either way), the two architectures differ in a way that silently produces near-total false negatives - this model is bidirectional with sliding-window attention, while gpt_oss is causal/decoder-style. That mismatch doesn't error out; it just quietly predicts "O" (no entity) almost everywhere, while looking numerically self-consistent. If you hit that, upgrade transformers instead of working around it.

Two things don't just fall out of a plain torch.onnx.export call, either way:

1. MoE routing. The expert-routing module has two forward code paths: a data-dependent, per-batch Python loop (.nonzero()-driven "which experts got hit") used at runtime, which isn't traceable for export; and a fully vectorized dense computation over all 128 experts (weighted by mostly-zero routing weights), which is traceable and mathematically identical. This export monkeypatches the routing module to always use the dense path. Getting this exactly right matters: the routing weights arrive as a sparse (num_tokens, top_k) pair (router_indices, routing_weights), not a dense (num_tokens, num_experts) tensor, so they have to be scattered into a dense per-expert weight matrix first; and this checkpoint's gate/up projection uses a concatenated split (gate, up = gate_up.chunk(2, dim=-1)), not the interleaved split (gate_up[..., ::2]/[..., 1::2]) some sibling MoE architectures use. Either bug produces an export that's internally self-consistent (matches its own incorrect PyTorch reference to ~1e-5) but wrong relative to the true model - only comparing against a real forward pass on a known example (e.g. does it actually tag "John Smith" as a name?) catches it.
2. Precision. The checkpoint's config.json declares a default of bf16. Loading in that precision and exporting produces Where nodes with bf16 scalar operands that onnxruntime's CPUExecutionProvider doesn't have a kernel for (a NOT_IMPLEMENTED error at session-load time, not export time). This export explicitly loads and exports in full fp32.

Verification

The exported graph was validated against a native, unpatched PyTorch forward pass of the original checkpoint on the same inputs (both loaded fp32), running on onnxruntime's CPUExecutionProvider:

• Max absolute logit difference (ONNX vs. true native PyTorch): ~1.8e-5
• Spot-checked against a plain-English sentence containing a name, phone number, and email - all three correctly tagged with their entity types and BIOES boundaries.

Usage

import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("lemonade-sdk/openmed-privacy-filter-multilingual-v2-onnx")
session = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

text = "My name is John Smith, call me at 555-123-4567 or email [email protected]."
encoded = tokenizer(text, return_tensors="np")
(logits,) = session.run(None, {
    "input_ids": encoded["input_ids"],
    "attention_mask": encoded["attention_mask"],
})  # logits: [batch, seq_len, 217] (BIOES over 54 entity types + "O")

No custom mask construction needed - the graph takes a plain 2D input_ids + attention_mask, same convention as a standard BERT-family token classifier.

Using this with the Lemonade router

This repo is self-contained for Lemonade's onnxruntime backend (via ort-server) - model.onnx + model.onnx.data + tokenizer.json + config.json + manifest.json all sit together at the repo root, so it can be registered directly as a checkpoint with no extra file staging.

1. Register the classifier:

curl -X POST http://localhost:13305/v1/pull -H "Content-Type: application/json" -d '{
  "model_name": "user.privacy-filter-ml-v2-onnx",
  "checkpoint": "lemonade-sdk/openmed-privacy-filter-multilingual-v2-onnx",
  "recipe": "onnxruntime"
}'

2. Register the router policy - a ready-to-use collection.router policy that routes to a local model whenever any of this classifier's 216 non-"O" labels crosses min_score: 0.5, and to a cloud model otherwise, is published separately at lemonade-sdk/pii_policy_openmed-privacy-filter-multilingual-v2-onnx:

hf download lemonade-sdk/pii_policy_openmed-privacy-filter-multilingual-v2-onnx --local-dir .
curl -X POST http://localhost:13305/v1/pull -H "Content-Type: application/json" \
  --data-binary @pii_policy_openmed-privacy-filter-multilingual-v2-onnx.json

The policy's routing.candidates (Qwen3.5-0.8B-GGUF local / fireworks.kimi-k2p6 cloud) and min_score are starting points, not fixed requirements - swap either to whatever local/cloud models you have configured. On the Nemotron-PII benchmark (20,000 PII-bearing prompts), this policy scored a 0% leak rate (zero PII prompts routed to the cloud candidate).

For building your own routing policy from scratch, or a more sophisticated one (multiple classifiers, LLM-based routing, custom match logic) rather than adapting this one, see the lemonade-router-builder skill - it turns a natural-language description of routing intent into a valid collection.router policy JSON.

License

Apache 2.0, inherited from the base model.

Citation

Please cite the original model and its underlying work:

@misc{openmed-privacy-filter-v2,
  title  = {OpenMed Privacy Filter Multilingual v2},
  author = {OpenMed},
  year   = {2026},
  url    = {https://huggingface.co/OpenMed/privacy-filter-multilingual-v2}
}

See the original model card for the complete BibTeX entry crediting OpenMed, OpenAI, and the AI4Privacy contributors.