distilbert-base-uncased-cfpd

DistilBERT for CFPB Consumer Complaint Classification

This model is a fine-tuned version of distilbert-base-uncased for classifying consumer complaint narratives from the Consumer Financial Protection Bureau (CFPB) consumer complaints dataset.

The model performs multi-class classification across seven consolidated consumer-finance categories and is intended to support automated complaint routing and triage.

Model Description

• Base model: distilbert-base-uncased
• Task: Multi-class text classification
• Domain: Consumer financial complaints
• Language: English
• Training framework: Hugging Face Transformers
• Training samples: 14,198
• Number of output classes: 7

Intended Uses & Limitations

Intended Uses

The model is intended for:

• Automated classification of consumer complaint narratives
• Complaint routing and triage
• Supporting downstream analysis of consumer financial complaints
• Demonstrating transformer-based NLP classification in an applied setting

The model is intended as a decision-support component and should not replace human review where classification accuracy is critical.

Limitations

Performance may vary for:

• Complaints that differ substantially from the training data
• Ambiguous or incomplete narratives
• Very short complaint descriptions
• Categories with relatively few training examples
• Data from domains or populations that differ from the CFPB dataset

The model was trained on CFPB consumer complaint data and may not generalize reliably to other domains without further evaluation or fine-tuning.

Training and Evaluation Data

The model was trained using consumer complaint narratives from the Consumer Financial Protection Bureau (CFPB) consumer complaints dataset.

The original CFPB complaint categories were consolidated into seven broader operational categories:

The training dataset contains 14,198 samples. The validation dataset is used to monitor model performance during training and for the reported evaluation metrics.

Preprocessing

Complaint narratives were cleaned before being provided to the model.

The preprocessing included:

• Separating punctuation from surrounding text
• Removing non-alphanumeric characters
• Replacing CFPB-scrubbed sequences such as XXXX with the token mask
• Removing repeated whitespace
• Removing hyperlinks where present

The original text casing was retained during preprocessing to allow the pretrained DistilBERT tokenizer and model to handle the input.

Training Procedure

The model was fine-tuned using the Hugging Face Transformers framework.

Training Hyperparameters

• Learning rate: 5e-05
• Training batch size: 16
• Evaluation batch size: 8
• Number of epochs: 4
• Random seed: 42
• Optimizer: AdamW (PyTorch fused)
• Betas: (0.9, 0.999)
• Epsilon: 1e-08
• Learning rate scheduler: Linear

Training Performance

Training completed after 1,005.7 seconds (approximately 16.8 minutes).

The final training run produced:

• Training loss: 0.4490
• Training samples/second: 56.47
• Training steps/second: 3.532
• Total training steps: 3,552
• Epochs completed: 4

Evaluation Results

The following results were obtained on the evaluation set:

Training Results

Validation performance improved through the first three epochs, with the highest validation F1 score of 0.8109 achieved at epoch 3. Performance declined slightly at epoch 4, while training loss continued to decrease.

Usage

The model can be loaded using the Hugging Face Transformers library:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_name = "appcle/distilbert-base-uncased-cfpb"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

For inference, provide a consumer complaint narrative to the tokenizer and pass the resulting inputs to the model.

Framework versions

• Transformers 5.15.0
• Pytorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2

Disclaimer

This model is provided for research, development, and demonstration purposes. Reported metrics are based on the evaluation data used during model development and should not be interpreted as a guarantee of performance on new or different datasets.