hs-roberta-v2-onnx

hs-roberta-v2-onnx

A 3-class English hate-speech / offensive-language classifier. RoBERTa fine-tuned with LoRA on the Davidson dataset, then merged and exported to ONNX (fp32 + dynamic int8) so it's small enough to run entirely in the browser via transformers.js.

Live demo: https://karpit0499.github.io/hate-speech-adv-v2/ (classification runs client-side — text never leaves the device)

Labels

id2label (Davidson ClassLabel order):

Model description

• Base model: unitary/unbiased-toxic-roberta (a RobertaForSequenceClassification with 16 multi-label toxicity heads).
• Adaptation: LoRA fine-tuning with the 16-head classifier swapped for a fresh 3-class head (ignore_mismatched_sizes=True), LoRA applied to the query and value projections, and the new classifier head trained via modules_to_save.
• Export: merged LoRA weights → ONNX (Optimum), with dynamic int8 quantization, arranged into the onnx/ subfolder that transformers.js expects. Quantized weights are ~125 MB (which is why this lives on the Hub, not in a GitHub repo).

This repo is the ONNX export. The full-precision PyTorch model is at kmrarpit2704/hs-roberta-v2.

How to use

Browser / Node (transformers.js):

import { pipeline } from '@huggingface/transformers';

const classify = await pipeline('text-classification', 'kmrarpit2704/hs-roberta-v2-onnx');
const result = await classify('put your sentence here');
console.log(result); // [{ label: 'offensive', score: 0.97 }]

Python (ONNX Runtime via Optimum):

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model = ORTModelForSequenceClassification.from_pretrained("kmrarpit2704/hs-roberta-v2-onnx")
tokenizer = AutoTokenizer.from_pretrained("kmrarpit2704/hs-roberta-v2-onnx")

clf = pipeline("text-classification", model=model, tokenizer=tokenizer)
print(clf("put your sentence here"))

Training data

tdavidson/hate_speech_offensive (Davidson et al., 2017) — ~25k English tweets labelled hate speech / offensive language / neither. Split roughly 80/10/10 (stratified), with a held-out test set of 2,479 examples.

Evaluation

Reported on the held-out Davidson test set (2,479 examples).

Confusion matrix (rows = true, columns = predicted):

                 pred:hate  pred:off  pred:neither
true:hate           90         38          15
true:offensive     112       1752          55
true:neither        12         21         384

Limitations & bias

• hate speech is the weak class (F1 0.504). It is both under-detected (recall 0.629) and over-predicted (precision 0.421), with most of the error being confusion with offensive — the well-known hate↔offensive boundary problem in this dataset. Do not rely on this model to reliably catch hate speech in isolation.
• The offensive class is profanity-driven. Because the Davidson offensive label is defined largely by swearing, profanity-free insults tend to be classified as neither. This is expected behaviour, not a bug, but it means the model is a poor fit for "polite" harassment.
• Documented dataset bias. Models trained on the Davidson dataset are known in the literature to over-flag African American English and in-group/reclaimed language as offensive or hateful. Expect this failure mode.
• English-only, short-text, tweet-domain. Accuracy drops on longer text, formal registers, or other domains and languages.
• Not a moderation decision-maker. Treat the output as a signal for human review, not an automated verdict, and never as the basis for punitive action.

License

Released under the MIT License.