modernbert-large-consumer-finance-11cls

ModernBERT-large — Consumer Finance Complaint Classification (11 labels)

Fine-tuned ModernBERT-large (~395M params) for multi-class classification of US consumer finance complaints into 11 consolidated categories.

Context

This model was developed as part of Project 12 — "Compare AI Algorithms: Machine Learning vs. LLM" of the OpenClassrooms AI Developer certification by William Derue.

The project scenario involves ZenAssist, a customer support platform serving 200+ companies. The goal is to automatically label incoming consumer complaints to route them to the correct support department, comparing traditional ML approaches with LLM-based inference.

This fine-tuned encoder model represents the supervised ML approach — a single forward pass through ModernBERT produces classification logits, making it fast, cheap to run, and suitable for high-throughput production deployment.

Labels

The original dataset contains 18 raw product tags, consolidated into 11 categories to reduce semantic overlap and improve model performance:

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="WillisBack/modernbert-large-consumer-finance-11cls",
    truncation=True,
    max_length=1024,
)

result = classifier("I was charged an overdraft fee on my checking account without prior notice.")
print(result)
# [{'label': 'Bank account', 'score': 0.95}]

Training

Class weights

Computed with sklearn.utils.class_weight.compute_class_weight("balanced") to handle class imbalance:

Results

Per-class performance

Observations

• Strong classes (F1 ≥ 0.70): Mortgage, Student loan, Credit reporting, Debt collection, Bank account, Credit card, Money transfer — these cover ~94% of the evaluation set.
• Weak classes: Consumer Loan, Payday loan, Vehicle loan or lease suffer from semantic overlap (all are loan products) and low sample counts.
• Other financial service (4 eval samples, 244 train samples) remains unlearnable at this scale. Consider merging with the nearest class or removing for production.
• Early stopping triggered at epoch 1.6 — the model converged before completing 2 full epochs.

Dataset

Trained on WillisBack/dataset-financial-user-claim — a cleaned, deduplicated, and label-consolidated version of the US CFPB Consumer Complaints dataset.

Files

Limitations

• Trained on English-language US CFPB complaints only. Performance on other languages or domains is unknown.
• Tail classes (Other financial service, Consumer Loan, Vehicle loan or lease) have low F1 — predictions on these should be treated with lower confidence.
• Max input length is 1024 tokens. Longer complaints are truncated.

Citation

@misc{derue2026modernbert-finance,
  author = {Derue, William},
  title = {ModernBERT-large Fine-tuned for Consumer Finance Complaint Classification},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/WillisBack/modernbert-large-consumer-finance-11cls}
}