consumer-finance-complaints-cfpb-adapter

Consumer Finance Complaints — CFPB Taxonomy Adapter

LoRA adapter for hierarchical classification of CFPB consumer complaint narratives, fine-tuned on Mixtral-8x7B-Instruct-v0.1 via Adaption's AutoScientist platform.

Given a complaint narrative, the adapter emits a three-level label from the CFPB taxonomy: Product → Issue → Sub-issue, where each level is constrained to be valid under its parent.

---

Task

This is closed-label classification, not open generation. The output space is a fixed taxonomy, not free text:

The unconstrained space would be 11 × 86 × 209 ≈ 197,714 combinations. The taxonomy's hierarchy reduces that to 393 valid triples — a 503× reduction. A correct output is one of those 393; anything else is a label hallucination.

Output format is three lines, verbatim labels, no explanation:

Product: <label>
Issue: <label>
Sub-issue: <label>

---

Evaluation results

Training win rates (Adaption internal)

A +66 point improvement on the in-domain evaluation.

Cross-domain tradeoff (catastrophic forgetting). The adapter specializes at the cost of general capability. On the Legal reference category, win rate drops from 64% (base) → 36% (adapted). This is expected for a task-specific adapter and is reported for transparency, not hidden.

Held-out evaluation (this is the comparable number)

The win rate above is Adaption's internal metric. The numbers below are measured on a temporal holdout the adapter never saw during training (742 rows, July 2025), scored with the same parser and the same 393-triple registry used to evaluate every other method.

The adapter's value proposition is not only accuracy but label validity: a closed-label task where the base model hallucinates 21.6% of the time, and where the adapter — having seen the 393 valid triples during training — should drive that toward zero. Fill this table before publishing.

Train/eval metrics

Validation loss tracked train loss without divergence across all 5 epochs, confirming generalization rather than memorization — important given the small (393) output space.

---

Model details

Inference note. data_format was chat and train_on_inputs was false. To reproduce results, wrap the narrative as a user turn via the tokenizer's chat template — do not feed raw text. Mismatched formatting measures format error, not model quality.

---

Training dataset

Fernandosr85/adaption-consumer-finance-complaints

12,346 prompt/completion pairs, stratified by (Product, Issue) pair. Built from the full CFPB complaint dump with a deliberate methodology:

Methodological note on the eval window. CFPB narrative publication collapses from August 2025 (intake continues, publication does not) and is effectively zero by 2026. Any naive temporal holdout on recent CFPB data is contaminated by this. The July 2025 window is the last month at the stable publication plateau.

---

Known limitations

• Completion regeneration. Adaptive Data regenerates completions on every run; a freeform Blueprint cannot enforce membership in a closed label set. This adapter was trained on the original (verbatim) completions, not the platform-enhanced ones, which contained retired labels (e.g. Bank account or service, removed by the CFPB in August 2023). Downstream users of the platform on closed-label tasks should map the original completion column.
• Single-month eval. Held-out evaluation is one month (July 2025) for the reasons above. A replication on June 2025 would strengthen the temporal-robustness claim.
• Domain specialization. See the Legal cross-domain drop above. This adapter is not intended for general-purpose use.

---

Model repositories

---

Credits

• Fine-tuning platform: Adaption — AutoScientist & Adaptive Data
• Challenge: AutoScientist Challenge 2026 — Personal Finance category
• Data source: CFPB Consumer Complaint Database (public)
• Author: Fernando Rodrigues · Kaggle: fernandosr85 · HuggingFace: Fernandosr85

---

Disclaimer

Experimental research artifact submitted to the AutoScientist Challenge 2026 (Personal Finance category). Classification outputs are automated and require review before any operational use.