rubert-tiny2-russian-emotion-detection-quantized

rubert-tiny2-russian-emotion-detection · quantized

Quantized builds of Aniemore/rubert-tiny2-russian-emotion-detection — multi-label emotion recognition for Russian text over seven classes: anger, disgust, enthusiasm, fear, happiness, neutral, sadness.

The weights here are the published original, quantized. They were not retrained and they are not a different model.

Variants

<img src="assets/quality.svg" alt="Quality after quantization" width="760"> <img src="assets/size.svg" alt="Weights on disk" width="760">

ROC AUC is listed first because the head is multi-label: macro-F1 depends on the decision threshold, which is 0.5 here because that is what the head was trained under, while ROC AUC does not.

How much this actually saves

Only Linear layers are quantized. In a BERT classifier the embedding matrix is not one of them, and on the smaller models it is most of the checkpoint — so the saving here scales with the encoder rather than with the parameter count. The large model compresses well; rubert-tiny barely moves, and the table above says so rather than quoting a ratio from the layers that did shrink.

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo = "Aniemore/rubert-tiny2-russian-emotion-detection-quantized"
model = AutoModelForSequenceClassification.from_pretrained(
    repo, subfolder="int8").eval()          # or "fp8", "int4"
tok = AutoTokenizer.from_pretrained(repo, subfolder="int8")

x = tok("мне сегодня очень грустно", return_tensors="pt")
with torch.no_grad():
    # multi-label: sigmoid per class, not softmax over classes
    probs = model(**x).logits.sigmoid()[0]
print({model.config.id2label[i]: round(p.item(), 3) for i, p in enumerate(probs)})

Limitations

• Weight-only, round-to-nearest, no calibration.
• Scored on the CEDR-m7 test split only. CEDR is written text; performance on transcribed speech, which carries no punctuation and no casing, is not measured here.
• Inherited from cointegrated/rubert-tiny2; the licence follows the base model.