flan-t5-small-safety-judgment

flan-t5-small-safety-judgment

1. Project Overview

This repository contains a fine-tuned version of the Flan-T5-Small model, specifically optimized for detecting Prompt Injection attacks. The model acts as a security guardrail, classifying incoming user prompts as either Safe or Injection.

---

2. Model Performance

The following metrics were derived from a test dataset evaluation. The model focuses on high sensitivity to ensure malicious inputs are intercepted before reaching the core LLM.

Confusion Matrix

Key Metrics

• Accuracy: 82.69%
• Precision: 82.53%
• Recall (Sensitivity): 86.12%
• F1-Score: 84.29%

Analysis

The model demonstrates a strong Recall of 86.12%, which is critical for security tasks. While there is a False Positive rate of ~21% (193 cases), this "strict" posture is often preferred in production environments to prevent bypasses, even at the cost of occasional over-blocking.

---

3. Disclaimers & Bias Statement

Disclaimer

Important: This model is provided "as-is" for research and supplementary security purposes. Prompt injection is an evolving adversarial field; therefore, this model should not be your only line of defense. It is highly recommended to use this in conjunction with output filtering, structural delimiters, and least-privilege API permissions.

Dataset & Potential Bias

• Pattern Bias: The training data may be biased toward common injection strings (e.g., "Ignore previous instructions"). It may underperform on novel, obfuscated, or multi-step "jailbreak" attempts.
• Linguistic Bias: The model was primarily trained on English-language datasets. Performance on multilingual prompts or code-heavy inputs (e.g., Python/SQL injection) may vary and should be validated separately.
• Contextual False Positives: The model may flag legitimate administrative commands as injections if the phrasing closely mimics adversarial patterns.

Technical Limitations

As a "Small" model (approx. 80M parameters), Flan-T5-Small may lack the deep semantic nuance of larger models like T5-Base or Large, but it offers significantly lower latency and resource consumption for real-time monitoring.