nigeria-loan-classifier-v2

Nigeria Loan Classification Closed-Label Interpreter v2

Author: Hussein Adeiza (mabera) Role: Licensed Environmental Health Officer, Abuja Nigeria Base Model: Llama 4 Scout 17B-16E Fine-tuned with: AutoScientist by Adaption Labs

Model Description

A LoRA adapter fine-tuned for closed-label loan classification, combining individual loan classification by days-past-due (Performing/Substandard/ Doubtful/Lost) with bank-level NPL ratio compliance classification (Compliant/Breach), against real CBN Prudential Guidelines and current Nigerian banking sector data.

Honest Disclosure: Weak Win Rate and a Suspected Platform Pattern

This model's win rate is below 50%, meaning the base model outperformed the adapted model on both the training-set win rate and the General Win Rate. This is disclosed transparently rather than omitted.

What was tested: this dataset was deliberately scaled from an initial 9 rows to 20 rows, using additional real, cited banking data, specifically to test whether insufficient data volume explained the weak result. It did not: the win rate was 40%/60% (adapted/base) at 9 rows and remained 40%/60% at 20 rows, with the General Win Rate at 45%/55%, essentially unchanged despite more than doubling verified, correct training data.

Suspected cause: this result fits a pattern observed across four submissions in this author's portfolio. Submissions that the platform's automatic classifier assigns to Science domain train on Llama 3.3 70B and show strong win rates (e.g. this author's AMR Classifier v2: 58% on-dataset, 71% General Win Rate). Submissions classified into News, Governance, or Corporate-business domains train on Llama 4 Scout 17B and consistently show weaker results, this model included. This dataset was intended for the Personal Finance category but was auto-classified as Corporate-business. This pattern was reported to the Adaption team (bugs channel) prior to publishing this model.

What was independently confirmed correct: the weak win rate is not attributable to dataset quality or labeling errors. All 20 rows were programmatically verified against CBN Prudential Guidelines ground truth, including boundary tests at all four classification threshold transitions (89/90, 179/180, 359/360 days) and a genuine near-miss NPL ratio case (4.99%, one hundredth of a percentage point below the 5% breach threshold). See the accompanying Kaggle notebook.

Training Data

• Source: CBN Prudential Guidelines for Deposit Money Banks, 2010 (as amended); Nairametrics, Businessday NG, and ThisDayLive reporting, 2024-2025
• Dataset: 20 rows (11 individual loan classifications + 9 bank/industry NPL ratio classifications), no recipe modifications applied
• Kaggle: https://www.kaggle.com/datasets/yunusahusseinadeiza/nigeria-loan-classification-interpreter

Training Metrics

• Win rate (on dataset): 40% adapted vs 60% base model
• General Win Rate: 45% adapted vs 55% base
• Base model: meta-llama/Llama-4-Scout-17B-16E
• Method: LoRA, zero recipe modifications, system-level closed-label constraint
• Dataset quality: 8.0 → 8.4 (+5.0% relative improvement, Grade B)
• Domain classification: Corporate-business (100%), not cleanly matched to the intended Personal Finance category

Key Cited Findings (from verified source data only)

• Industry-wide NPL ratio breached the CBN's 5% threshold in April 2025 (5.62%), with 11 banks in breach, up from 6 a year earlier
• GTCO Holdings (5.18%) and UBA (6.64%) both breached individually in 2024
• An independent 8-bank average sat at 4.99%, one hundredth of a percentage point below breach, illustrating how close the sector came to a wider compliance failure even where the aggregate remained technical
• The 2009 historic crisis peak (37.3%) provides context for how far current figures remain from genuine systemic crisis levels

Credits

Powered by Adaptive Data — Adaption Labs AutoScientist Challenge 2026, Part 2 — Personal Finance Category