sme-risk-classifier-distilbert

SME Risk Classification Model (DistilBERT)

A fine-tuned DistilBERT model designed to classify Small and Medium Enterprises (SMEs) into Medium Risk or High Risk categories based on structured financial and managerial indicators.

---

🧠 Model Summary

This model takes structured SME data (financial literacy, financial risk, risk assessment, management ability, financial analysis, and SME metadata) and converts it into a text representation suitable for transformer-based classification.

It was fine-tuned using:

• Base model: distilbert-base-uncased
• Task: Binary text classification
• Labels:
  • 0 → Medium Risk
  • 1 → High Risk

---

📊 Dataset

Dataset Source:
Harvard Dataverse — SME Financial Decision-Making Dataset (2023)
(Republished via Kaggle — CC0 Public Domain)

Dataset Size: 443 total SMEs

• Train: 310
• Validation: 66
• Test: 67

Features include:

• FL1–FL4 (Financial Literacy)
• FR1–FR4 (Financial Risk)
• RA1–RA4 (Risk Assessment)
• MDA1–MDA4 (Management Decision Ability)
• FDM1–FDM4 (Financial Decision-Making)
• FA1–FA4 (Financial Analysis)
• SME metadata: Sector, Type, Size, Established Year

A risk_score was computed and used to derive the binary risk label.

---

🔧 Training Details

• Model: DistilBERTForSequenceClassification
• Epochs: 4
• Batch size: 8
• Learning rate: 5e-5
• Weight decay: 0.01
• Optimizer: AdamW
• Hardware: CPU-based training (no GPU)

The model was trained using the Hugging Face Trainer API.

---

📈 Evaluation Results

Validation Set

• Accuracy: 1.00
• Weighted F1: 1.00

Test Set

• Accuracy: 1.00
• Weighted F1: 1.00

Confusion Matrix (Test Set)

The model performs strongly, though results reflect limited examples of high-risk SMEs.

---

🚀 How to Use

Load the model:

from transformers import DistilBertForSequenceClassification, DistilBertTokenizerFast
import torch

model = DistilBertForSequenceClassification.from_pretrained("Harry1001/sme-risk-classifier-distilbert")
tokenizer = DistilBertTokenizerFast.from_pretrained("Harry1001/sme-risk-classifier-distilbert")

text = "Established_year: 3 | Sector: 4 | FL1: 3 | RA1: 4 | FA2: 2 | MD1: 3"

inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

pred = torch.argmax(logits, dim=1).item()
id2label = {0: "Medium", 1: "High"}
print(id2label[pred])