distilbert-support-classifier

DistilBERT Customer Support Ticket Classifier

Fine-tuned distilbert/distilbert-base-uncased for classifying customer support tickets into 27 intent categories.

Model Description

This model classifies raw customer support ticket text into one of 27 issue-type intents, enabling automated routing, prioritisation, and analytics for customer support pipelines.

• Architecture: DistilBERT-base-uncased (66M parameters, 40% smaller and 60% faster than BERT-base while retaining 97% of its performance)
• Task: Multi-class text classification (27 classes)
• Training data: Bitext Customer Support LLM Chatbot Training Dataset — 26,872 English utterances, nearly perfectly balanced across all classes (imbalance ratio: 1.05×)

Supported Intent Classes

Training Configuration

Evaluation Results

⏳ Model training pending. Accuracy and Macro F1 will be filled in after training completes.

Evaluated on a held-out test set (15% of data, ~4,031 samples):

See confusion_matrix.png for the full per-class breakdown (added after training).

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="annebanne/distilbert-support-classifier",
)

# Single ticket
result = classifier("I need to cancel my order, it hasn't shipped yet.")
print(result)
# [{'label': 'cancel_order', 'score': 0.98}]

# Batch of tickets
tickets = [
    "Where is my refund? It's been 2 weeks.",
    "I can't log into my account after resetting my password.",
    "Please send me an invoice for order #12345.",
    "My payment keeps getting declined.",
]
results = classifier(tickets)
for ticket, res in zip(tickets, results):
    print(f"{res['label']:35s} ({res['score']:.2%}) — {ticket[:60]}")

Dataset

• Source: bitext/Bitext-customer-support-llm-chatbot-training-dataset
• Size: 26,872 utterances
• Language: English
• Balance: Near-perfect — each class has ~950–1,000 examples (imbalance ratio: 1.05×)
• Split: 85% train (22,841) / 15% test (4,031), random seed 42

Limitations

• Trained on synthetic/augmented data. Real-world distribution may differ from production tickets.
• Performs best on short, single-intent utterances (similar to training data style).
• English only.
• Does not handle multi-intent tickets (predicts a single label).

Citation

If you use this model, please cite the base model:

@article{sanh2019distilbert,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},
  journal={arXiv preprint arXiv:1910.01108},
  year={2019}
}

<!-- ml-intern-provenance -->

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

• Try ML Intern: https://smolagents-ml-intern.hf.space
• Source code: https://github.com/huggingface/ml-intern