distilbert-toxic-en

Content Moderation Model

Fine-tuned DistilBERT for real-time toxic content detection. Built as a research prototype exploring automated content moderation with a planned extension to Hindi and Hinglish language moderation.

Model Details

Performance

Labels

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="Void10/distilbert-toxic-en",
    return_all_scores=True
)

result = classifier("You are such a wonderful person!")
# [{'label': 'LABEL_0', 'score': 0.97}, {'label': 'LABEL_1', 'score': 0.03}]

Training Details

Fine-tuned on a 20K subsample of the SetFit toxic conversations dataset using HuggingFace Trainer API on a T4 GPU via Google Colab. Labels were binarized from multi-label annotations into clean (0) vs toxic (1).

Training Code

from transformers import (DistilBertTokenizerFast,
                          DistilBertForSequenceClassification,
                          TrainingArguments, Trainer)

args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=32,
    per_device_eval_batch_size=64,
    warmup_steps=200,
    evaluation_strategy="epoch",
    fp16=True,
)

Limitations

• Class Imbalance: ~10% of training samples are toxic, causing conservative flagging and lower recall (60%). Addressable via weighted loss or oversampling.
• English Only: This model was trained exclusively on English text. Performance degrades significantly on Hindi, Hinglish, or other Indic languages.
• Domain: Trained on social media style text. May not generalize well to formal documents or domain-specific content.
• Bias: Like all models trained on human-annotated data, this model may reflect annotator biases around identity groups and cultural context.

Research Extension (Planned)

The primary limitation of existing content moderation research — including recent work like SLM-Mod (NAACL 2025) — is the exclusive focus on English data. This model serves as the English baseline for a planned multilingual extension targeting:

• Hindi toxic comment detection
• Hinglish (code-mixed Hindi-English) moderation
• Base model: MuRIL (Google, 2021) or IndicBERT (AI4Bharat)
• Target dataset: HASOC Hindi Abusive Comment Dataset

Live Demo

Try the model live at:
huggingface.co/spaces/Void10/content-moderation-demo

Citation

If you use this model in your research, please cite:

Author

B.Tech Computer Science (AI/ML) · Shivalik College of Engineering · Dehradun, India
Built as a college research minor project · 2026