spot-distilbert-phishing

spot-distilbert-phishing

A DistilBERT-based binary email classifier that labels emails as phishing or legitimate. This is the model shipped inside SPOT's analyzer-nlp plugin; it is mirrored here as a public artefact so researchers, integrators, and auditors can inspect what SPOT actually runs in production.

Model description

• Architecture: DistilBertForSequenceClassification (6 layers, 768-dim hidden state, 12 attention heads). Sequence-classification head with two output labels.
• Base checkpoint: cybersectony/phishing-email-detection-distilbert_v2.1, re-headed from 4 labels to 2 (ignore_mismatched_sizes=True) and fine-tuned end-to-end.
• Tokenizer: WordPiece, 30 522 tokens, max sequence length 512.
• Parameters: ~67 million.
• Output: a 2-logit softmax [P(legitimate), P(phishing)]. The predicted class is the argmax.

Intended use

• Primary use case: phishing detection on inbound email bodies in the SPOT platform. The analyzer-orchestrator calls POST /internal/analyze on analyzer-nlp; the wrapper in analyzer-nlp loads this model and answers an AnalyzerVerdict (a score, a confidence and the nlp.phishing_probability signal) that the workflow's verdict policy combines with the other analyzers' answers.
• Suitable inputs: English emails, ideally up to 512 tokens: the subject and the body, concatenated and tokenised together. In SPOT, the analyzer feeds it the subject, a space, then the message's visible text (the text part, or the text the HTML part shows).
• Out of scope: SPOT does not use this model in isolation. The workflow combines it with rule-based, contextual, and (optionally) LLM-based analyzers; the final phishing verdict is the orchestrator's aggregate, not this model's argmax. Using the raw classifier on its own will produce more false positives and false negatives than the end-to-end SPOT pipeline.

Training data

Fine-tuned on the public SetFit/enron_spam dataset:

Labels: 0 = legitimate, 1 = phishing/spam. The dataset is the Enron corpus relabelled by SetFit; it skews toward early-2000s English business email and reflects historical phishing patterns. Performance on contemporary phishing (modern brand impersonation, AI-generated text, image-only emails) is not guaranteed by this corpus, which is exactly why SPOT augments the verdict with rule-based and contextual analyzers.

Training procedure

Hardware: 2x NVIDIA Tesla T4 GPUs, run as a Kaggle notebook. Framework: Hugging Face transformers.Trainer.

Evaluation

Final metrics on the held-out SetFit/enron_spam test split (2 000 examples) at the end of epoch 3:

Accuracy / precision / recall / F1 were not computed during training. The production wrapper in analyzer-nlp/tests/ exercises the end-to-end SPOT integration on a larger, internal evaluation set; those numbers are not yet published here.

How to use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tok = AutoTokenizer.from_pretrained("spotproject/spot-distilbert-phishing")
model = AutoModelForSequenceClassification.from_pretrained("spotproject/spot-distilbert-phishing")

text = "Dear customer, your account has been suspended. Click here to verify..."
inputs = tok(text, return_tensors="pt", truncation=True, max_length=512)
with torch.no_grad():
    logits = model(**inputs).logits
probs = torch.softmax(logits, dim=-1)[0]
print({"legitimate": probs[0].item(), "phishing": probs[1].item()})

For the full SPOT integration ; with workflow context, knowledge-store enrichment, and aggregation across analyzers ; see analyzer-nlp on Codeberg.

Limitations and biases

• Domain skew. Trained on Enron-era English business email. Expect degraded performance on consumer/mobile messaging, on non-English text, on modern AI-generated phishing, and on email-as-a-document attack vectors (HTML smuggling, image-only emails, QR-code phishing).
• Fuzzy class boundary. The dataset's positive class collapses spam, marketing, and phishing into a single label. Treating the model's "phishing" probability as a calibrated phishing-only score will be optimistic.
• No adversarial robustness guarantees. Simple rephrasing or Unicode obfuscation can flip the verdict. SPOT mitigates this by combining the model with non-NLP analyzers in the workflow orchestrator.
• Personal data. Do not feed PII into a hosted version of this model without consent. The SPOT deployment runs the analyzer inside the customer's perimeter for that reason.

License

Apache-2.0, matching the analyzer-nlp source repository.

Citation

@software{spot_distilbert_phishing,
  title  = {spot-distilbert-phishing: phishing classifier for the SPOT platform},
  author = {SPOT Project},
  year   = {2026},
  url    = {https://huggingface.co/spotproject/spot-distilbert-phishing},
  license = {Apache-2.0}
}

Related

• Analyzer wrapper (production code): https://codeberg.org/SPOT_Project/analyzer-nlp
• SPOT platform: https://codeberg.org/SPOT_Project/core
• Project home: https://codeberg.org/SPOT_Project