Downloads · 30 days
7
14% of all-time downloads
LRieser/steam-ukraine-political-mmbert-base
steam-ukraine-political-mmbert-base is a text classification model from LRieser. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
Binary text classifier for Steam game reviews written around a developer's public statement on the Russia-Ukraine war (spring 2022). It decides whether a review engages the political content of that statement or is or…
Downloads · 30 days
7
14% of all-time downloads
All-time downloads
49
Public
Parameters
308M
1.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.2 GB · 97%
From the Hugging Face model README
Binary text classifier for Steam game reviews written around a developer's public statement on the Russia-Ukraine war (spring 2022). It decides whether a review engages the political content of that statement or is ordinary product discussion. Fine-tuned from jhu-clsp/mmBERT-base on 40,011 reviews labeled by a large language model. Multilingual; the review is classified in its original language.
| Label | Meaning |
|---|---|
non_political | ordinary product discussion: gameplay, bugs, features, value, recommendations |
political | the review references the war, Ukraine, Russia, the developer's political stance, sanctions, geopolitics, nationalism, boycotts, or any other sociopolitical dimension, including meta-commentary on review bombing or "politics in games" |
The model returns a probability for political. The argmax decision
(threshold 0.50) favours recall; a threshold of 0.92, calibrated on a
development split, balances precision and recall and reproduces the
teacher's share of political reviews. Both are reported below.
The model was trained on the review text preceded by a short context cue, the reviewer's recommendation and whether the review was posted before or after the developer's statement:
[positive review, post-announcement] <review text>
[negative review, pre-announcement] <review text>
Use the same prefix at inference. The review text is the original-language text with whitespace normalised; no translation is needed.
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
repo = "LRieser/steam-ukraine-political-mmbert-base"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
texts = [
"[negative review, post-announcement] Uninstalled after the devs decided to lecture us about the war.",
"[negative review, pre-announcement] Crashes every ten minutes since the last patch, refunded.",
]
@torch.inference_mode()
def p_political(texts, batch_size=64):
out = []
for i in range(0, len(texts), batch_size):
enc = tok(texts[i:i + batch_size], padding=True, truncation=True,
max_length=256, return_tensors="pt").to(model.device)
out.extend(model(**enc).logits.softmax(-1)[:, 1].tolist())
return out
probs = p_political(texts)
labels = ["political" if p >= 0.92 else "non_political" for p in probs]
pipeline("text-classification", model=repo) also works and applies the
argmax (0.50) decision.
jhu-clsp/mmBERT-base (ModernBERT architecture, about 307M
parameters), sequence-classification head with two labels.deepseek-chat, temperature 0.2) prompted with a written codebook and
five few-shot examples. Political share 13.7%. Largest
languages: English 41%, Russian 13%, Simplified Chinese 11%, Polish 7%, Turkish 4%, German 4%.Test set of 4,446 teacher-labeled reviews (609 political, 13.7%), disjoint from training. Thresholds were fixed on the early-stopping split before the test set was scored once.
| Metric | Threshold 0.50 (argmax) | Threshold 0.81 | Threshold 0.92 (recommended) |
|---|---|---|---|
| Agreement with teacher | 0.9550 | 0.9573 | 0.9591 |
| Political precision | 0.792 | 0.826 | 0.853 |
| Political recall | 0.911 | 0.872 | 0.847 |
| Political F1 | 0.847 | 0.848 | 0.850 |
| Macro-F1 | 0.9105 | 0.9117 | 0.9132 |
| Predicted political share | 15.8% | 14.5% | 13.6% |
Confusion matrix at 0.92 (rows: true, columns: predicted):
| non_political | political | |
|---|---|---|
| non_political | 3,748 | 89 |
| political | 93 | 516 |
Political-class F1 by review language (languages with at least 50 test reviews, 10 of them political):
| Language | n | political | F1 at 0.50 | F1 at 0.92 |
|---|---|---|---|---|
| English | 1,808 | 252 | 0.885 | 0.878 |
| Russian | 469 | 35 | 0.658 | 0.769 |
| Chinese | 441 | 168 | 0.879 | 0.869 |
| Polish | 233 | 46 | 0.857 | 0.882 |
| German | 197 | 21 | 0.857 | 0.870 |
| Korean | 109 | 15 | 0.727 | 0.667 |
| Ukrainian | 57 | 10 | 0.762 | 0.762 |
Scoring the 4,446-review test set takes about 12 seconds on an RTX
4090. The companion generative classifier
(LRieser/steam-ukraine-political-qwen2.5-3b-merged)
reaches macro-F1 0.9168 on the same test set in about 20 minutes.
Full metrics, including the threshold sweep, are in
eval_results.json.
@misc{rieser2026steamukrainepoliticalmmbert,
author = {Rieser, Lars and Furneaux, Brent},
title = {Steam political-relevance classifier (mmBERT-base)},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/LRieser/steam-ukraine-political-mmbert-base}
}