nsfw_text_detection

Model Card: DistilBERT for NSFW Content Detection

Model Description

This model is a fine-tuned DistilBERT-base-uncased model designed for binary classification of text content as either safe (0) or potentially Not Safe For Work (NSFW) (1). It leverages the power of transformer-based architectures for natural language understanding to identify patterns indicative of NSFW content.

Intended Use

The model is intended to assist in content moderation tasks, specifically for flagging text posts that may contain NSFW material. It can be used as a preliminary filter to reduce human reviewer workload or to enforce content policies on platforms.

Training Data

The model was trained on a proprietary dataset.

• Total Data Points : 197,912
• Class Distribution:
  • Safe (0): 86.79%
  • NSFW (1): 13.21%

Model Architecture and Fine-tuning

• Base Model: DistilBERT-base-uncased
• Tokenizer: AutoTokenizer from 'distilbert-base-uncased'

Commercial Inference API

This model is available via a high-availability, low-latency API for production use cases on RapidApi

Limitations

• Language: The model is based on 'distilbert-base-uncased', primarily designed for English text.

Usage Instructions

from transformers import pipeline

repo = 'acaciabengo/nsfw_text_detection'

classifier = pipeline("text-classification", model=repo)

# Get all scores 

all_scores = classifier("This is some text.", top_k=None)

nsfw_score = next(item['score'] for item in all_scores if item['label'] == 'LABEL_1')

print(f"NSFW Probability: {nsfw_score}")