PhilBERT

PhilBERT: Phishing Detection with DistilBERT

PhilBERT is a fine-tuned DistilBERT model optimized for detecting phishing threats across multiple communication channels, including emails, SMS, URLs, and websites. It is trained on a diverse dataset sourced from Kaggle, Mendeley, Phishing.Database, and Bancolombia, ensuring high adaptability and real-world applicability.

---

Key Features

• Multi-Channel Detection – Analyzes text, URLs, and web content to detect phishing patterns.
• Fine-Tuned on Real-World Data – Includes recent three months of financial institution data (Bancolombia).
• Lightweight & Efficient – Based on DistilBERT, providing high performance with reduced computational costs.
• High Accuracy – Achieves 85.22% precision, 93.81% recall, and 88.77% accuracy on unseen data.
• Self-Adaptive Learning – Continuously evolves using real-time phishing simulations generated with GPT-4o.
• Scalability – Designed to support 7,000–25,000 simultaneous users in production environments.

---

Model Architecture

PhilBERT leverages DistilBERT, a distilled version of BERT, maintaining the same architecture but with 40% fewer parameters, making it lightweight while preserving high accuracy. The final model includes:

• Tokenizer: Trained to recognize phishing-specific patterns (URLs, obfuscation, domain misspellings).
• Custom Classifier: A fully connected dense layer added for binary classification (phishing vs. benign).
• Risk Scoring Mechanism: A weighted confidence score applied to enhance detection reliability.

---

Data Preprocessing

Before fine-tuning, the dataset underwent extensive preprocessing to ensure balance and quality:

• Duplicate Removal & Balancing: Maintained a near 50-50 phishing-to-benign ratio to prevent model bias.
• Feature Extraction: Applied to URLs, HTML, email bodies, and SMS content to enrich input representations.
• Dataset Split: Final dataset included:
  • 427,028 benign URLs & 381,014 phishing URLs
  • 17,536 unique email samples
  • 5,949 SMS samples
  • Web entries filtered for efficiency (removing entries >100KB).
• Export Format: Data transformed and stored in JSON for efficient training.

---

Training & Evaluation

PhilBERT was fine-tuned on multi-modal phishing datasets using transfer learning, achieving:

• False Positive Reduction: Multi-layered filtering minimized false positives while maintaining high recall.
• Scalability: Successfully stress-tested for up to 25,000 simultaneous users.
• Compliance: Meets ISO 27001 and GDPR standards for security and privacy.

---

Usage

Installation

pip install transformers torch

Inference

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "your_username/PhilBERT"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "Click this link to update your bank details: http://fakebank.com"
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)

print(f"Phishing probability: {predictions[0][1].item():.4f}")

---

License

This model is proprietary and protected under a custom license. Please refer to the LICENSE file for terms of use.

---