content-multilabel-iab-classifier

Fine-tuned LoRA Classifier on MiniLM for IAB Multi-Label Classification

This is a fine-tuned LoRA (Low-Rank Adaptation) classifier based on MiniLM (microsoft/MiniLM-L12-H384-uncased), designed for multi-label content classification using the IAB content taxonomy. The model can assign one or more categories to input text — making it suitable for tasks such as content classification.

🔍 Model Details Model Description

This model is based on microsoft/MiniLM-L12-H384-uncased, a compact and efficient transformer model optimized for fast inference and low memory footprint. It has been fine-tuned using LoRA for multi-label classification over 20 IAB categories plus an "inconclusive" fallback class.

The model predicts multiple applicable content labels from:

inconclusive
animals
arts
autos
business
career
education
fashion
finance
food
government
health
hobbies
home
news
realestate
society
sports
tech
travel

Key Configuration:

Base Model: microsoft/MiniLM-L12-H384-uncased
Task: Multi-label content classification
Label Count: 21 (multi-hot vector)
Language: English
Fine-tuning Method: PEFT with LoRA
LoRA Config:
    r=16
    lora_alpha=16
    lora_dropout=0.1
    target_modules=["query", "key"]
Developed by: Mozilla
License: Apache-2.0

📦 Model Sources: Demo (optional): Hugging Face Space

📥 Usage:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained("Mozilla/content-multilabel-iab-classifier")
tokenizer = AutoTokenizer.from_pretrained("Mozilla/content-multilabel-iab-classifier")

label_list = [
    'inconclusive',
    'animals',
    'arts',
    'autos',
    'business',
    'career',
    'education',
    'fashion',
    'finance',
    'food',
    'government',
    'health',
    'hobbies',
    'home',
    'news',
    'realestate',
    'society',
    'sports',
    'tech',
    'travel'
]
label2id = {label: idx for idx, label in enumerate(label_list)}
id2label = {idx: label for label, idx in label2id.items()}

text = "Discover the latest trends in AI and wearable technology."

with torch.no_grad():
    inputs = tokenizer(text, return_tensors="pt", truncation=True)
    outputs = model(**inputs)
    probs = torch.sigmoid(outputs.logits).squeeze().cpu().numpy()
    predicted_labels = [(id2label[i], round(p, 3)) for i, p in enumerate(probs) if p >= 0.5]

📖 Citation

If you use this model, please cite it as:

@misc{mozilla_iab_multilabel_lora,
  title       = {Fine-tuned LoRA Classifier on MiniLM for IAB Multi-Label Classification},
  author      = {Mozilla},
  year        = {2025},
  url         = {https://huggingface.co/mozilla/content-multilabel-iab-classifier},
  license     = {Apache-2.0}
}