credit-default-prediction-randomforest-galcohen

Credit Card Default Prediction — Machine Learning Pipeline

Data Science Course —Project
Author: Gal Cohen video: https://www.youtube.com/watch?v=1-XZN4MaLoE

Project Overview

This project builds a complete machine learning pipeline for predicting credit card default risk.
It includes full EDA, cleaning, feature engineering, clustering, model comparisons, and exporting the best model.

---

Visual Overview

Below are some of the key visuals generated throughout the analysis.

Default Distribution

---

Credit Limit Distribution

---

---

K-Means Clusters (PCA 2D Visualization)

---

Random Forest Feature Importance (Top 15)

---

ROC Curves — Before vs After Feature Engineering

---

Dataset

Source: UCI Default of Credit Card Clients dataset
Rows: 30,000 originally → 28,070 after cleaning
Target: DEFAULT

---

Data Cleaning

Key cleaning operations:

• Removed negative bill amounts & negative payments (~1,930 rows)
• Checked & imputed missing values after feature engineering
• Cleaned invalid categorical codes
• Verified no duplicates

---

Exploratory Data Analysis (EDA)

A detailed exploration was performed including:

• Distributions (histograms, boxplots)
• Default behavior across demographics
• Correlation heatmap
• Delay behavior (PAY_0…PAY_6)
• Bill/payment trends
• PCA & clustering insights

---

Feature Engineering

Created multiple new predictive features:

Aggregation Features

TOTAL_BILL, TOTAL_PAY, BILL_PAY_RATIO

Delay Features

AVG_DELAY, MAX_DELAY

Trend

BILL_TREND

Clustering (K-Means)

• CLUSTER_ID
• Distances to centroids (DIST_K0…DIST_K4)
• DIST_TO_CENTROID

These features significantly improved model performance.

---

Regression → Classification (Part 7)

Used linear regression predictions to create a continuous target, then applied a median split, resulting in a perfectly balanced classification target:

---

Models Trained

Baseline

• Linear Regression
• Logistic Regression

With Engineered Features

• Logistic Regression
• SVM
• Random Forest Classifier

---

Winning Model: Random Forest

Reasons for winning:

• Captures non-linear patterns
• Handles complex feature interactions
• Performs well with engineered features
• Lowest error & highest ROC-AUC