DeepSeek-Distill-Multiclass-HateSpeech

DeepSeek-Distill-Multiclass-Hate-Speech

DeepSeek-Distill-Multiclass-Hate-Speech is a distilled large-scale language model specialized in analyzing social media content related to hate speech categories. The base model provides a strong contextual representation learning, which improves performance on hate speech detection.

• Base Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
• Language: English
• Model Type: Causal decoder-only transformer fine-tuned for classification using supervised learning.

Primary Use Cases

• Moderation of user-generated text
• Automated labeling for hate speech datasets
• Research on abusive language detection
• Further fine-tuning on more data and labels

Supported Labels

Model Inference

You can use the model directly on any dataset. To use it, follow the following steps:

1. Install the required libraries Ensure that you have installed the libraries before using the model using pip:

pip install transformers torch bitsandbytes accelerate

2. Load the Model and Tokenizer

# Import required Modules
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

# Load model and Tokenizer using 4 bit for efficient memory usage
model_name = 'hugsanaa/DeepSeek-Distill-Multiclass-Hate-Speech'
model = AutoModelForCausalLM.from_pretrained(
    model_name, 
    device_map={"": 0},
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.float16,
        ),
    )

tokenizer = AutoTokenizer.from_pretrained(model_name,
                                          trust_remote_code=True,
                                          max_length=512,
                                          padding_side="left",
                                          add_eos_token=True,
                                          )
tokenizer.pad_token = tokenizer.eos_token

3. Predict

# Example of Cyberbullying text
labels_list = [
    "Immigration and Xenophobic Hate Speech",
    "Gender-Based Hate Speech",
    "Racial and Ethnic Hate Speech",
    "Profanity and General Abuse",
    "Religious Hate Speech",
    "Non-hate"
]

text = "Illegal Alien Hit Man, Dragged Him For Blocks In Deadly DUI Crash #BuildTheWall #NoAmnesty #NoDACA #DeportThemAll #KeepAmericaSafe"

prompt = f"""
You are an AI model specialized in classifying text into categories of hate speech or identifying if it contains no hate.
Below is a text sample. Determine the most appropriate category it belongs to.
Possible categories: {", ".join(labels_list)}.
Choose one of these exact categories as your answer.

Text: [{text}]
Answer:
"""

message = [
    {"role": "user", "content": prompt},
]

# Tokenize input
inputs = tokenizer.apply_chat_template(
    message,
    add_generation_prompt=False,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)


# Make Predictions
outputs = model.generate(**inputs, max_new_tokens=20, top_p=0.9,
                        do_sample=False, temperature=0.6,
                        pad_token_id=tokenizer.eos_token_id,)

# Interpret results
y_pred = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]).strip()
print(f"Prediction: {y_pred}")