kotodama-distilbert-emotion

Kotodama — DistilBERT emotion classifier

distilbert-base-uncased fine-tuned on dair-ai/emotion with class-weighted cross-entropy, for the Kotodama project.

Why macro-F1 rather than accuracy

The dataset is heavily imbalanced — joy is 9.4x the size of surprise. A model that predicted only joy would score 33.5% accuracy while being useless. The headline metric is therefore macro-F1, which weights the rarest class exactly as heavily as the most common one, and training used class weights inversely proportional to class frequency.

Results (held-out test split)

The gain is concentrated in the minority classes — exactly the ones a raw-accuracy comparison would let a model ignore.

Labels

Index order matters — it is the order of the model's logits.

Usage

from transformers import pipeline

classifier = pipeline("text-classification", model="AsifZaman1912/kotodama-distilbert-emotion", top_k=None)
classifier("i cannot believe this actually happened")

Training

Checkpoint selection was on validation macro-F1, never validation loss: with a weighted objective the two can disagree, and loss favours models that have conceded the minority classes. The test split was used exactly once, for the numbers above.

ONNX

onnx/model.onnx is an fp32 export. onnx/model.int8.onnx is dynamically quantized — 4x smaller and faster, but it measures -0.0078 macro-F1 against fp32, so fp32 remains the serving default.