roberta-base-category-type-generator-28k

Model Details

This model is designed to classify customer service inquiries into five categories: Technical Issues, Billing & Payment, Product Inquiries, Account Management, and Policy Questions.

Training Data

The model was trained on a balanced dataset of 28000 entries composed of anonymized customer service inquiries. Each category contained a similar number of examples to prevent class imbalance. https://github.com/amosproj/amos2023ws01-ticket-chat-ai/tree/main/Backend/app/model/test_data/test_data_with_gpt

Training Procedure

The model was fine-tuned over four epochs for a sequence classification task. We utilized a batch size of 4 and an Adam optimizer with a learning rate of 2e-5.

Model Performance

The model's performance was evaluated using a confusion matrix and a learning curve, as detailed below:

• Confusion Matrix Analysis

  

• Learning Curve Analysis

  

  • Training Loss: This line starts at approximately 1.2 and steadily decreases to about 0.6, suggesting that the model is learning and improving its ability to classify the training data correctly.
  • Validation Loss: The validation loss begins around 0.9 and decreases to just under 0.7 after three epochs. The validation loss mirrors the training loss's downward trend, which is a good sign that the model is generalizing well and not overfitting the training data.