emotion-qwen3-label-gen

Emotion Qwen3 Label Generation (Instruction)

A PEFT (LoRA) adapter for Qwen/Qwen3-0.6B that generates multi-label emotion tags from short text (e.g. tweets). Given a tweet, the model outputs a comma-separated list of emotions from a fixed set of 11 labels.

Model Details

Model Description

• Developed by: shreevershith
• Model type: Causal language model with LoRA adapter for text-to-label generation
• Language(s) (NLP): English
• Finetuned from model: Qwen/Qwen3-0.6B
• License: Same as base model (refer to Qwen3-0.6B)

Model Sources

• Repository: emotion_detection_using_transformers_kaggle — notebook: QLoRA_Qwen3_Label_Generation_InstructModel_Final.ipynb

Uses

Direct Use

Use this model to predict which emotions (from a fixed set of 11) are present in a short English text. Input: tweet or similar text. Output: comma-separated emotion labels, which you can map to a binary vector over the 11 emotions.

Emotion labels: anger, anticipation, disgust, fear, joy, love, optimism, pessimism, sadness, surprise, trust.

Downstream Use

Suitable for dashboards, APIs, or pipelines that need emotion tags from user-generated text (e.g. social media, feedback). Parse the generated string into the 11-dimensional binary vector for scoring or visualization.

Out-of-Scope Use

• Not intended for non-English text without further evaluation.
• Not a substitute for clinical or legal sentiment/emotion analysis.
• May reflect biases present in the training data (tweets).

Bias, Risks, and Limitations

The model was fine-tuned on tweet-style data and may not generalize well to formal text, other domains, or other languages. Predictions can be subjective and should not be used as the sole basis for high-stakes decisions.

Recommendations

Use for exploration and demos; validate on your own data before production use.

How to Get Started with the Model

Install dependencies:

pip install torch transformers peft

Load the base model and adapter, then run generation:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# ----------------------
# Setup
# ----------------------
MODEL_NAME = "Qwen/Qwen3-0.6B"
ADAPTER_NAME = "shreevershith/emotion-qwen3-label-gen"

device = "cuda" if torch.cuda.is_available() else "cpu"

# Load base model and adapter
base_model = AutoModelForCausalLM.from_pretrained(MODEL_NAME)
model = PeftModel.from_pretrained(base_model, ADAPTER_NAME)
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_NAME)

# Make sure we have a pad token
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model.eval()
model.to(device)

label_cols = [
    "anger", "anticipation", "disgust", "fear", "joy", "love",
    "optimism", "pessimism", "sadness", "surprise", "trust",
]

def extract_emotions(text: str):
    return list({
        e.strip().lower()
        for e in text.lower().split(",")
        if e.strip() in label_cols
    })

# ----------------------
# Example inference
# ----------------------
test_tweet = "I'm so happy and grateful today!"
prompt = f"Tweet: {test_tweet}\n### LABEL:"

inputs = tokenizer(
    prompt,
    return_tensors="pt",
    truncation=True,
    max_length=384,
).to(device)

with torch.no_grad():
    # IMPORTANT: use sampling; Qwen3 docs say don't use greedy
    outputs = model.generate(
        **inputs,
        max_new_tokens=20,
        pad_token_id=tokenizer.pad_token_id,
        do_sample=True,
        temperature=0.7,
        top_p=0.8,
        top_k=20,
    )

# Only decode the newly generated tokens
input_len = inputs["input_ids"].shape[1]
new_tokens = outputs[0][input_len:]
generated_text = tokenizer.decode(new_tokens, skip_special_tokens=True)

print("Full generated continuation:", repr(generated_text))

label_str = generated_text.split("### LABEL:")[-1].strip()
print("Label string:", repr(label_str))

emotions = extract_emotions(label_str)
binary_labels = [1 if label in emotions else 0 for label in label_cols]

print("Detected emotions:", emotions)
print("Binary vector:", dict(zip(label_cols, binary_labels)))

Training Details

Training Data

• Source: Kaggle competition Emotion Detection Spring 2025 — train.csv with tweet text and 11 binary emotion columns.
• Splits: 7,724 labeled samples; 80/20 train/validation (6,179 train, 1,545 validation).
• Preprocessing: URLs removed; @mentions normalized to @user; hashtag symbol removed (text kept); optional synonym-based data augmentation (nlpaug) for minority classes.

Training Procedure

• Method: QLoRA / LoRA (PEFT) on Qwen/Qwen3-0.6B; gradient checkpointing enabled.
• Task: Causal LM; model is prompted with the tweet and generates the label sequence (e.g. joy, optimism).
• Sequence length: max_length=384.
• LoRA config: rank r=8, lora_alpha=16, lora_dropout=0.05; target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.

Training Hyperparameters

Speeds, Sizes, Times

• Environment: Google Colab (GPU).
• Adapter size: ~20.2 MB (adapter_model.safetensors).

Evaluation

Testing Data, Factors & Metrics

• Validation set: 1,545 held-out samples from the same Kaggle train split.
• Metrics: Per-class F1 (binary, threshold-optimized on validation); macro F1 and accuracy over the 11 emotion labels.

Results

Validation per-class F1 (from threshold optimization in the notebook):

Submissions were made to Kaggle Emotion Detection Spring 2025; this instruction-tuned adapter produced more calibrated, selective predictions than the base-model adapter.

Summary

The model performs best on high-frequency emotions (e.g. joy, anger, fear, disgust) and weaker on rarer ones (e.g. trust, surprise, anticipation), consistent with class imbalance in the training data.

Technical Specifications

Model Architecture and Objective

• Base: Qwen3-0.6B (decoder-only causal LM)
• Adapter: LoRA (PEFT), task type CAUSAL_LM
• Output: Generated text (comma-separated emotion names); post-processing maps to an 11-dimensional binary vector.

Compute Infrastructure

Hardware

• Training: Google Colab (GPU runtime, typically T4 or similar).

Software

• Frameworks: PyTorch, Transformers, PEFT 0.15.2, TRL, bitsandbytes, Accelerate
• Data & logging: datasets, wandb, scikit-learn, pandas, nlpaug
• Python: 3.11 (Colab default)

Citation

If you use this model, please cite the base model (Qwen3) and the PEFT library as appropriate.

Model Card Contact

For questions or issues, open a discussion on the Hugging Face model page or an issue in the project repository.