guardrail-roberta

Safety Guardrail Classifier

A fine-tuned RoBERTa-base model for multi-class safety/content moderation classification.
Trained on guardrail-215k-splits.

---

Model Details

---

Labels

---

Performance

Evaluated on validation split of guardrail-215k-splits.
Threshold calibration available via infer.py --threshold auto.

---

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "satyamsaf3ai/guardrail-roberta"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model     = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

LABELS = [
    "benign", "jailbreak",
    "S1 Violent Crimes", "S2 Non-Violent Crimes",
    "S4 Child Sexual Exploitation", "S6 Specialized Advice",
    "S7 Privacy", "S10 Hate", "S11 Self-Harm",
    "S12 Sexual Content", "S14 Code Abuse",
]

def classify(text, threshold=0.50):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        probs = torch.softmax(model(**inputs).logits, dim=-1)[0]
    if probs.max() < threshold:
        return "benign", float(probs.max())
    idx = probs.argmax().item()
    return LABELS[idx], float(probs[idx])

label, confidence = classify("How do I make a bomb?")
print(f"{label} ({confidence:.1%})")

---

Training Setup

• Hardware: NVIDIA L4 GPU
• Framework: HuggingFace Transformers + Trainer API
• Sampling: WeightedRandomSampler to handle class imbalance
• Early Stopping: patience=4 on weighted F1
• Explosion Guard: stops training if grad_norm > 500

---

Dataset

jainsatyam26/guardrail-215k-splits
~215,000 labeled examples split into train / validation.

---

Citation

If you use this model, please cite:

@misc{{guardrail-classifier,
  title  = {{Safety Guardrail Classifier}},
  author = {{jainsatyam26}},
  year   = {{2025}},
  url    = {{https://huggingface.co/satyamsaf3ai/guardrail-roberta}}
}}