yhile-toxic-lr-v1

yhile-toxic-lr-v1

A logistic regression classifier that predicts whether text is hate speech, offensive, or neither. Built with scikit-learn using TF-IDF text features combined with hand-engineered structural features targeting identity-based exclusion language.

This README was written with AI assistance.

Model details

• Algorithm: Logistic Regression (scikit-learn), multiclass
• Features: TF-IDF word 1-2-grams (10,000 features) + character 3-5-grams (5,000 features), combined with engineered features: identity/protected-group term detection, exclusionary-language detection ("get out," "go back to," "don't belong here"), dehumanizing-language detection, and a combined identity+exclusion signal aimed at catching structural hate-speech patterns that pure keyword/slur detection misses
• Format: saved with skops (not pickle) for safe loading
• License: CC-BY-SA 3.0 (inherited from the Stormfront training data — see Training data below; this is a stronger requirement than a plain permissive license, see Licensing note)

Training data

Final combined dataset: ~194,000 rows. Class balance: ~79% neither, ~19% offensive, ~1.4% hate — hate speech is a genuine minority class in all three source datasets.

Licensing note: the Stormfront dataset is licensed CC-BY-SA 3.0, a share-alike license. Because this trained model incorporates that data, this repository is released under the same CC-BY-SA 3.0 license rather than MIT, to respect the share-alike requirement of the source data.

Performance

Evaluated on a held-out 20% stratified test split (~38,840 rows):

• Accuracy: 91.1%
• Neither: 0.97 precision / 0.93 recall
• Offensive: 0.81 precision / 0.84 recall
• Hate: 0.22 precision / 0.63 recall The hate class numbers are the most important to understand honestly — see Limitations below.

Usage

from skops.io import load, get_untrusted_types
 
untrusted = get_untrusted_types(file="yhile-toxic-lr-v1.skops")
saved = load("yhile-toxic-lr-v1.skops", trusted=untrusted)
 
model = saved["model"]
word_vectorizer = saved["word_vectorizer"]
char_vectorizer = saved["char_vectorizer"]
label_names = saved["label_names"]  # {0: "hate", 1: "offensive", 2: "neither"}
 
# see train.py for the full extract_features() implementation used at
# both training and inference time — the two must stay in sync

The full training pipeline and feature engineering are included as train.py in this repository for transparency.

Limitations

This model has real, tested limitations — documented here rather than glossed over:

• Hate speech precision is low (0.22). Roughly 3 out of 4 things the model flags as "hate" are actually something else (usually "offensive"). The model is tuned to catch more real hate speech at the cost of more false alarms — a real precision/recall tradeoff, not a bug.
• Indirect hate speech without explicit identity keywords is often missed — e.g. "that entire group of people is ruining this neighborhood" with no named group. The engineered features rely on identity terms being present in the text.
• Diminishment-based hate speech (e.g. certain forms of misogyny — "women should stay quiet," "that gender can't be trusted") is not well covered. This is a distinct linguistic pattern from the identity+exclusion pattern this model was specifically strengthened for, and testing confirmed the model does not reliably catch it.
• Sarcasm is not reliably detected, consistent with known limitations of bag-of-words text classifiers generally.
• Some ground-truth labels in the training data are themselves contested — the Davidson dataset's line between "hate speech" and "offensive language" has been publicly critiqued as inconsistent (e.g. slur-containing tweets are frequently labeled "offensive" rather than "hate" by the original annotators). This model inherits that ambiguity.
• Rare spurious false positives occur from coincidental word-level correlations typical of bag-of-words models (e.g. a message using the word "breed" in an unrelated context was flagged) — these are not systematic, but they are a known category of error.
• English only.
• This model was not evaluated on live, real-world moderation traffic — only on held-out samples from its own training distribution.

Ethical note

The Stormfront dataset consists of sentences from an actual white supremacist forum, sampled and labeled for research purposes to help build counter-hate-speech tools. The Wikipedia dataset contains real, sometimes-hostile comments from actual Wikipedia editors. Both are used here for their stated research purpose: building tools to detect and reduce hateful and toxic content, not to reproduce or endorse it.

Attribution

If you use this model or its training data, please cite the original dataset authors:

Davidson, T., Warmsley, D., Macy, M., Weber, I. Automated Hate Speech Detection and the Problem of Offensive Language. ICWSM 2017.

de Gibert, O., Perez, N., García-Pablos, A., Cuadros, M. Hate Speech Dataset from a White Supremacy Forum. ALW2, 2018.

Wulczyn, E., Thain, N., Dixon, L. Ex Machina: Personal Attacks Seen at Scale. WWW 2017.