Downloads · 30 days
25
38% of all-time downloads
kokuren/doku-chiwawa-ja-offensiveness-deberta-v2-small
doku-chiwawa-ja-offensiveness-deberta-v2-small is a text classification model from kokuren. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as cc-by-sa-4.0.
kokuren/doku-chiwawa-ja-offensiveness-deberta-v2-small is a Japanese social-media text model that predicts seven continuous offensiveness-related dimensions.
Downloads · 30 days
25
38% of all-time downloads
All-time downloads
65
Public
Parameters
17.9M
144 MB on disk
Likes
0
Public
Click a slice to open those files.
.onnx72.4 MB · 49%
From the Hugging Face model README
kokuren/doku-chiwawa-ja-offensiveness-deberta-v2-small is a Japanese social-media text model that predicts seven continuous offensiveness-related dimensions.
The model is fine-tuned from izumi-lab/deberta-v2-small-japanese, a Japanese DeBERTa-v2 Small model.
The base model is distributed under CC BY-SA 4.0, and this repository uses the same license.
The primary intended use is lightweight Japanese text analysis in Doku-Chiwawa, including browser-side inference through ONNX / Transformers.js.
Typical use cases:
This model is not a moderation authority, diagnosis system, legal classifier, or substitute for human review. Scores are model estimates and can be wrong, especially for irony, quotations, reclaimed language, fiction, dialect, unusual orthography, and context-dependent expressions.
The model has 7 logits. Apply a sigmoid to each logit to obtain a score in the range 0.0–1.0.
| Index | Key | Japanese description |
|---|---|---|
| 0 | insult | 侮辱 |
| 1 | threat | 脅迫 |
| 2 | obscene | 卑俗・下品表現 |
| 3 | identity_attack | 属性集団への攻撃 |
| 4 | sexual_explicit | 露骨な性的表現 |
| 5 | targetedness | 対象指向性 |
| 6 | indirect_hostility | 婉曲・間接的敵意 |
Doku-Chiwawa's application UI groups these predictions as follows:
insult, threat, identity_attack, and indirect_hostilityobscene and sexual_explicittargetedness is kept as an auxiliary model output and is excluded from those two summary circles.These groupings are application-level presentation rules, not additional outputs learned by the model.
izumi-lab/deberta-v2-small-japanesetokenizer.jsonThe tokenizer is intentionally browser-friendly: the repository includes a standard Hugging Face tokenizer.json, avoiding the external MeCab + UniDic preprocessing dependency used by an earlier Doku-Chiwawa prototype.
Training data: A locally prepared Japanese social-media text dataset. The raw training corpus is not included in this model repository.
Supervision: Seven continuous soft-target dimensions produced by an automated labeling pipeline and used for downstream fine-tuning.
The repository does not claim that these labels are human gold-standard annotations. Users should take the provenance and limitations of the supervision signal into account when interpreting model scores.
Metrics below are read automatically from the local test_metrics.json at upload time.
| Metric | Value |
|---|---|
| Mean MAE | 0.051422 |
| Mean RMSE | 0.089206 |
| Mean Pearson correlation | 0.568901 |
| Mean Spearman correlation | 0.605506 |
If labels are conceptually viewed on an original 0–4 scale, a normalized MAE can be multiplied by 4 for a rough scale conversion. The deployed model itself outputs normalized 0–1 scores after sigmoid.
| Dimension | Japanese | MAE | RMSE |
|---|---|---|---|
insult | 侮辱 | 0.060970 | 0.106174 |
threat | 脅迫 | 0.009144 | 0.027005 |
obscene | 卑俗・下品表現 | 0.039123 | 0.074403 |
identity_attack | 属性集団への攻撃 | 0.013566 | 0.045125 |
sexual_explicit | 露骨な性的表現 | 0.051872 | 0.094074 |
targetedness | 対象指向性 | 0.132556 | 0.183089 |
indirect_hostility | 婉曲・間接的敵意 | 0.052722 | 0.094570 |
See test_metrics.json for the complete machine-readable evaluation output.
The browser-oriented FP32 model is stored at:
onnx/model.onnx
The initial FP32 ONNX export is approximately 69 MB.
A dynamic INT8 experiment was substantially smaller (~25 MB), but showed unacceptable output drift relative to FP32 in a 500-example comparison and is therefore not included as a recommended artifact. A naive FP16 graph conversion also produced ONNX type-consistency problems and is not published here as a supported model.
FP32 is therefore the reference deployment artifact for this release.
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "kokuren/doku-chiwawa-ja-offensiveness-deberta-v2-small"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
model.eval()
text = "ここに解析したい日本語文章"
inputs = tokenizer(
text,
return_tensors="pt",
truncation=True,
max_length=192,
)
with torch.no_grad():
logits = model(**inputs).logits
scores = torch.sigmoid(logits)[0]
labels = ['insult', 'threat', 'obscene', 'identity_attack', 'sexual_explicit', 'targetedness', 'indirect_hostility']
print(dict(zip(labels, scores.tolist())))
The base model is not an unlimited-context model. Applications should chunk long input before inference.
The Doku-Chiwawa application may perform overlapping chunk inference and aggregate chunk scores. Such chunk aggregation is an application-side policy and is not part of the model itself.
config.json
model.safetensors
tokenizer.json
tokenizer_config.json
special_tokens_map.json
doku_chiwawa_meta.json
test_metrics.json
onnx/model.onnx
model.safetensors is the Transformers/PyTorch master artifact. onnx/model.onnx is the reference browser/deployment artifact.
targetedness was empirically more difficult to predict than several other dimensions during development.This fine-tuned model is released under Creative Commons Attribution-ShareAlike 4.0 (CC BY-SA 4.0), consistent with the base model license.
Base model: https://huggingface.co/izumi-lab/deberta-v2-small-japanese
Doku-Chiwawa is a lightweight Japanese text-analysis project designed for local/browser-assisted feedback interfaces.