privacy-filter-mlx

privacy-filter-mlx

On-device PII / secret detection for Apple Silicon (MLX / Metal). A repackaging of openai/privacy-filter (a gpt-oss-style 128-expert MoE token classifier) for the MLX runtime, used by the native Swift redactor pf.

It tags every token with one of 33 BIOES labels across 8 categories (account_number, private_address, private_date, private_email, private_person, private_phone, private_url, secret) so a downstream redactor can mask spans it has never seen before — names, emails, phone numbers, API keys, URLs — that exact-match maskers miss.

Variants

¹ Argmax labels matching the fp32 reference, measured on a 200-text / 40k-token synthetic PII eval set.

Quantization frontier (measured)

The MoE experts are ~90% of the weights, so they dominate the size/quality trade-off:

q4-8emb is the recommended default: a 69% size cut for ~0.5% label drift. 3-/2-bit are certified but risky for a redactor (label drift on a fail-closed task).

Usage

Download

# bf16 (root) — full fidelity, runtime-quantizable
hf download beshkenadze/privacy-filter-mlx --local-dir ./privacy-filter

# pre-quantized 870 MB only
hf download beshkenadze/privacy-filter-mlx --include "q4-8emb/*" --local-dir ./privacy-filter

Python (MLX)

The reference forward (pf_mlx.py) loads the bf16 weights and quantizes in-memory via a runtime knob:

PF_QBITS=4 PF_QEMBED=8 python pf_mlx.py        # 870 MB path
PF_QBITS=0 python pf_mlx.py                     # fp16, full size

Swift CLI (pf)

pf streams stdin → stdout and replaces detected spans with stable <CATEGORY_n> tokens, fail-closed (no raw value ever reaches stdout unless --fail-open):

cat app.log | pf --model ./privacy-filter
# Contact <PRIVATE_PERSON_0> at <PRIVATE_EMAIL_0>, key <SECRET_0>

Architecture

gpt-oss-style MoE token classifier: 8 layers, hidden=640, 14/2 attention heads × 64, intermediate=640, 128 experts top-4, attention sinks, bidirectional sliding-window attention (radius 128), interleaved YaRN RoPE (θ=150000, factor=32, truncate=false), o200k tokenizer, 33-label BIOES head. See config.json.

Why MLX, not Core ML / ANE

A 128-expert top-4 MoE needs a sparse gather, which the Apple Neural Engine cannot do — Core ML evicts the MoE to the GPU and runs it dense (32× redundant compute, ~336 ms). MLX's sparse gather_mm path on Metal is both exact and ~44× faster (~7.6 ms). For MoE on Apple Silicon, use MLX.

License & attribution

Apache-2.0, inherited from the upstream model. This repo is a format repackaging of openai/privacy-filter — all model credit to OpenAI. The bf16 weights at the root are byte-equivalent to the upstream safetensors; q4-8emb/ is a quantization of those weights.