diabetes-regression-model

🩺 Diabetes Risk Prediction — Regression & Classification

---

🎥 Presentation Video

<video src="https://huggingface.co/Danielhalali/diabetes-regression-model/resolve/main/assignment 2 video.mp4" controls="controls" style="max-width: 720px;"></video>

---

📌 Project Overview

This project was completed as part of a Data Science course assignment. The goal is to predict diabetes risk using lifestyle, demographic, and clinical data from nearly 100,000 individuals.

I explored the data, engineered new features, applied clustering, and trained multiple regression and classification models — comparing their performance and selecting the best one.

Research Question: Can we predict whether a person has diabetes based on their health and lifestyle data? And if so — which factors matter most?

---

📊 Dataset

• Source: Kaggle — Diabetes and Lifestyle Dataset
• Size: 97,297 rows × 31 columns
• Numeric features: Age, BMI, glucose levels, HbA1c, blood pressure, cholesterol, insulin, heart rate, waist-to-hip ratio
• Categorical features: Gender, ethnicity, education level, employment status, smoking status, income level
• Target (Regression): diagnosed_diabetes (0 = Healthy, 1 = Diabetic)
• Target (Classification): diabetes_risk_score converted to 3 balanced classes — Low Risk, Medium Risk, High Risk

---

🔍 EDA Highlights

Data Cleaning

• Identified missing values in columns such as smoking status, alcohol consumption, insulin level, and diabetes risk score
• Filled missing values using median for numeric columns and mode for categorical columns
• Applied Capping (Winsorization) using IQR to handle outliers in continuous features — without losing any data rows
• Note: Binary columns like family_history_diabetes and cardiovascular_history were excluded from capping since IQR would incorrectly flag the minority class as an outlier

Key Visualizations

Class Distribution: The dataset is relatively balanced — 59% of participants are diagnosed with diabetes and 41% are not. <img src="./pic1.png" width="600"/>

Age Distribution: Most participants are between ages 30 and 70, with a peak around age 50. The distribution gives a good demographic overview of the dataset and shows we have representation across a wide age range. <img src="./pic2.png" width="600"/>

Feature Correlation Heatmap: The heatmap shows how strongly each pair of features is related. Dark red means a strong positive relationship — when one goes up, the other goes up too. Fasting glucose and HbA1c are the darkest red relative to the diabetes diagnosis column, meaning they are the strongest predictors. Most other features appear light-colored, indicating weak direct correlations. <img src="./pic3.png" width="600"/>

Research Questions

1. Does BMI predict diabetes? Yes — the boxplot clearly shows that diabetic patients have a significantly higher median BMI than healthy individuals. This confirms that obesity is a key risk factor for diabetes in this dataset.

   <img src="./pic4.png" width="600"/>

2. Does smoking affect diabetes? Partially — across all smoking groups (Never, Former, Current), there are more diabetic individuals than healthy ones. However, the proportion doesn't change dramatically between groups, suggesting smoking alone is not the primary driver of diabetes in this dataset.

   <img src="./pic5.png" width="600"/>

3. Does fasting glucose increase with age? Yes — the scatter plot with a regression line shows a clear upward trend from left to right. As age increases, fasting glucose levels tend to rise as well. This helps explain why diabetes risk increases with age.

   <img src="./pic6.png" width="600"/>

---

🤖 Models

Baseline — Linear Regression (Part 3)

The baseline model was trained on the original features with default parameters.

The baseline model explains 92% of the data. Feature importance shows that diabetes_stage and HbA1c are the strongest predictors — both well-known medical risk factors for diabetes.

Actual vs Predicted: The plot shows predictions clustered around the binary values (0 and 1). The red dashed line represents a perfect prediction — the closer the points are to the line, the more accurate the model is. <img src="./pic13.png" width="600"/>

Residuals Distribution: The residuals plot shows 3 peaks — this is expected since the target is binary (only 0 or 1). The largest peak is centered around zero, meaning most predictions were close to the actual values. <img src="./pic14.png" width="600"/>

Feature Importance: The chart shows which features influenced the model most. diabetes_stage dominates — which makes sense since it is directly related to the diagnosis. HbA1c and family history of diabetes follow as the next most important predictors. <img src="./pic15.png" width="600"/>

---

⚙️ Feature Engineering

New Features Created

I created 5 new interaction features from existing columns:

All numeric features were then normalized using StandardScaler so that features with large values don't dominate the model over features with small values.

K-Means Clustering (k=4)

I used the Elbow Method to determine the optimal number of clusters — the graph shows a clear "elbow" at k=4, meaning adding more clusters beyond that doesn't significantly improve the groupings. <img src="./pic7.png" width="600"/>

I then visualized the clusters using PCA, which reduces all 31 dimensions to just 2 so we can draw a scatter plot. Each dot is one person, and each color is a different cluster. <img src="./pic8.png" width="600"/>

The clusters revealed 4 distinct health profiles:

The cluster ID and distance to centroid were added as new features for the models.

---

Improved Regression Models (Part 5)

After feature engineering and clustering, I retrained three models on the enriched dataset:

<img src="./pic9.png" width="600"/>

🏆 Regression Winner: Gradient Boosting (R² = 0.9988)

Random Forest and Gradient Boosting dramatically outperform Linear Regression because they can capture non-linear relationships between features — something Linear Regression cannot do. The engineered features (especially glycemic_load and glucose_insulin_ratio) appear in the top 15 most important features, confirming that feature engineering improved the models.

Classification Models (Part 8)

I converted the continuous diabetes_risk_score into 3 balanced classes using Quantile Binning (pd.qcut), each containing approximately 33% of the data. <img src="./pic16.png" width="600"/>

I focused on Recall as the key metric — in a medical context, a False Negative (missing a high-risk patient) is far more dangerous than a False Positive (incorrectly flagging a healthy person).

🏆 Classification Winner: Gradient Boosting (Accuracy = 0.94)

The confusion matrices show that Logistic Regression struggles most with the Medium Risk class, often confusing it with Low or High Risk. Gradient Boosting has significantly fewer off-diagonal errors across all classes. <img src="./pic10.png" width="600"/> <img src="./pic11.png" width="600"/> <img src="./pic12.png" width="600"/>

---

🎁 Bonus Work

Interactive Visualization (Plotly)

An interactive scatter plot showing fasting glucose vs BMI colored by cluster, with point size representing HbA1c level. Hovering over each point reveals age and diabetes diagnosis. The plot clearly shows that higher glucose and BMI correspond to the high-risk clusters. <img src="./bonus1.png" width="600"/>

Cross Validation

5-fold cross validation on a sample of 10,000 records:

• Mean Accuracy: 0.9301
• Std Deviation: 0.0028 (very low — confirms model stability)

All 5 folds scored between 0.928 and 0.934, confirming the model is consistent and not overfitted. <img src="./bonus2.png" width="600"/>

Learning Curve

The learning curve shows that as training size increases, validation accuracy steadily improves from 0.88 to 0.93. The training and validation curves are converging — confirming the model generalizes well and is not overfitting. Adding more data would likely continue to improve performance. <img src="./bonus3.png" width="600"/>

---

📁 Repository Contents

• Assignment_2.ipynb — Full Python notebook with all analysis, models, and visualizations
• regression_model.pkl — Winning regression model (Gradient Boosting, R² = 0.9988)
• classification_model.pkl — Winning classification model (Gradient Boosting, Accuracy = 0.94)
• Diabetes_and_LifeStyle_Dataset .csv — Dataset with manually introduced missing values for data cleaning practice