privacy-filter-ettin-32m

Sheltron Privacy Filter Ettin 32M

Fast, compact privacy filter for lightweight deployment.

Sheltron Privacy Filter Ettin 32M is a compact token-classification model for detecting privacy-risk spans in text, with same output format as OpenAI Privacy Filter. Model architecture is based on jhu-clsp/ettin-encoder-32m.

This model is intended for fast and light-weight usecases such as screening of emails, messages, conversations, and commands.

This model repository is standalone. Loading the root checkpoint requires only standard Hugging Face Transformers dependencies; loading an ONNX variant requires ONNX Runtime. The root AutoModelForTokenClassification.from_pretrained(...) call loads the FP32 Safetensors checkpoint; select an ONNX file explicitly when using FP32, FP16, or INT8 ONNX Runtime inference.

Model Details

The eight decoded categories are account_number, private_address, private_date, private_email, private_person, private_phone, private_url, and secret.

Although the base architecture exposes a longer positional limit, this model was trained and evaluated at 512 tokens. Inputs longer than 512 tokens should be processed with overlapping 512-token windows and stride 128.

Files And Runtime Targets

Usage

Transformers

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "sheltron-ai/privacy-filter-ettin-32m"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForTokenClassification.from_pretrained(model_id)
model.eval()

text = "Send the report to [email protected] after review."
encoded = tokenizer(
    text,
    return_tensors="pt",
    return_offsets_mapping=True,
    return_overflowing_tokens=True,
    truncation=True,
    max_length=512,
    stride=128,
    padding="max_length",
)
offsets = encoded.pop("offset_mapping")
encoded.pop("overflow_to_sample_mapping", None)

with torch.inference_mode():
    logits = model(**encoded).logits

The logits are BIOES token scores. Do not independently argmax them when reproducing the reported span metrics. Apply the constrained Viterbi transitions and operating point in viterbi_calibration.json, then convert decoded token offsets into character spans and deduplicate exact spans from overlapping windows.

ONNX Runtime

Install the ordinary ONNX inference dependencies; Optimum, bitsandbytes, and a custom runtime plugin are not required:

pip install onnxruntime transformers huggingface-hub numpy

Hugging Face does not define one loader shared by every INT8 format. This repository's INT8 artifact is an ONNX model, so download that file explicitly and open it with ONNX Runtime:

from huggingface_hub import hf_hub_download
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

model_id = "sheltron-ai/privacy-filter-ettin-32m"
filename = "onnx/model_quantized.onnx"  # or model.onnx / model_fp16.onnx
model_path = hf_hub_download(repo_id=model_id, filename=filename)

providers = ["CPUExecutionProvider"]
# For model_fp16.onnx use: providers = ["CUDAExecutionProvider"]
session = ort.InferenceSession(model_path, providers=providers)
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)

tokens = tokenizer(
    "Send the report to [email protected] after review.",
    return_tensors="np",
    truncation=True,
    max_length=512,
    padding="max_length",
)
input_names = {item.name for item in session.get_inputs()}
inputs = {
    key: np.asarray(value, dtype=np.int64)
    for key, value in tokens.items()
    if key in input_names
}
logits = session.run(None, inputs)[0]

Use viterbi_calibration.json for the same post-processing contract as the Transformers path. ONNX Runtime returns raw BIOES logits; constrained Viterbi decoding is required to reproduce the reported span metrics for every precision variant, not only INT8.

License And Attribution

The Sheltron Privacy Filter Ettin 32M model is released under Apache-2.0. The Ettin base model is published under the MIT License.