dlp-tier2-en-electra-auto-f2

English ELECTRA Tier2 DLP Classifier

This model is an English sequence classifier for the second stage of a DLP pipeline. It is fine-tuned from google/electra-small-discriminator on the EN2 hard-negative merged dataset.

Meaning of Tier 2

Here, "Tier 2" refers to the second stage of a DLP detection pipeline, not a model architecture or version.

This model is designed for text that Tier 1 has already flagged as a candidate or that policy requires for secondary review. It is not intended as a direct classifier for every arbitrary document; use it as the precision re-review stage after a rule-based first-stage filter.

Labels

The label names are descriptive metadata for the classification task. The training and inference code use integer labels.

Intended Use

Use this model to re-review text that a first-stage DLP filter flags or moves to a secondary review stage. It is designed for sentence- or paragraph-level risk classification, not for character-span extraction. Production use should combine the model with the pipeline's chunking, threshold, and policy logic.

Input and Output

Input

• A single UTF-8 English text string, typically one sentence or one chunk.
• The tokenizer requires no prompt template. Pass the text directly.
• Long documents should be split before inference. The recommended pipeline uses 128-character chunks with 32 characters of overlap, then evaluates each chunk independently.
• Each chunk is tokenized with truncation=True, padding=True, and max_length=256.
• Do not pass a task instruction, chat template, or conversation history.

Output

The model returns class logits for each input chunk:

• logits[0]: score for class benign (label ID 0)
• logits[1]: score for class leakage_risk (label ID 1)

Apply softmax to obtain class probabilities. The recommended positive-class probability is probabilities[1]. If it is greater than or equal to the decision threshold, classify the chunk as leakage_risk; otherwise classify it as benign.

For a multi-chunk document, the project pipeline reports the maximum chunk score as the document risk score and then applies the same threshold. The model itself does not aggregate multiple chunks.

Loading and Usage

• Required packages: transformers and torch.
• Load the tokenizer and classification model from the same repository or local directory.
• The model uses the standard AutoTokenizer and AutoModelForSequenceClassification interfaces; no trust_remote_code or custom Python code is required.
• Call model.eval() before inference and disable gradient calculation for production CPU/GPU inference.
• The recommended single-example setup uses truncation=True, padding=True, and max_length=256.
• For HF Hub loading, use the querypieai/dlp-tier2-en-electra-auto-f2 repository ID. For offline loading, use the local directory path containing this model card.

Recommended Inference Settings

• Decision threshold: 0.42 for the positive-class probability
• Max sequence length: 256
• Pipeline chunk size: 128
• Pipeline overlap: 32

The threshold was selected on validation data using the F2 objective with recall_min=0.9 and false_positive_rate_max=0.2. The threshold is not a model calibration parameter; it is a policy decision applied after probability calculation.

Training Configuration

• Base model: google/electra-small-discriminator
• Training data: EN2 hard-negative merged dataset
• Dataset availability: Private. This repository does not include the training data.
• Training/validation/test sizes: 25,553 / 3,708 / 3,751 records
• Positive (leakage_risk) ratio by split: train 7.06%, validation 10.17%, test 10.02%
• Text deduplication: SHA-256 after NFKC, whitespace normalization, and case folding
• Cross-split normalized-text and conversation-level leakage checks: all 0
• Optimization trials: 12
• Objective: F2 with recall and false-positive-rate constraints
• Learning rate: 3.317508234735466e-05
• Batch size: 8
• Epochs: 5
• Max length: 256
• Weight decay: 0.03
• Warmup ratio: 0.0

Evaluation

Test metrics at threshold 0.42:

Validation metrics at the selected threshold and final configuration:

The validation and test F1 values differ notably. Review split-level writing-style variation and group construction before using this checkpoint as a production gate.

Inference Example

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_id = "querypieai/dlp-tier2-en-electra-auto-f2"
threshold = 0.42

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Enter the text to review here."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256, padding=True)

with torch.no_grad():
    logits = model(**inputs).logits

positive_probability = torch.softmax(logits, dim=-1)[0, 1].item()
prediction = "leakage_risk" if positive_probability >= threshold else "benign"
print({"score": positive_probability, "prediction": prediction})

Limitations

• The model is a binary risk classifier and does not return PII entity offsets.
• Performance was measured on the project's EN2 validation and test sets; production performance may differ by domain, language style, and risk policy.
• False positives can occur on business messaging and obfuscated or partial identifiers.
• False negatives can occur for unusual identifiers or changed obfuscation patterns.
• Use human review or an additional adjudication stage for high-impact decisions.