exam-score-prediction-linear-regression

** Project Presentation Video** **click on the video **

<!-- 🎥 Project Presentation Video --> <p align="center"> <a href="https://youtu.be/UxKQZzwZom8?si=aAKqtFSHNHX99sy2" target="_blank"> <img src="https://img.youtube.com/vi/UxKQZzwZom8/0.jpg" alt="Exam Score Prediction – Project Presentation Video" width="480"> </a> </p>

this is the url(in case its not working):

https://youtu.be/UxKQZzwZom8?si=aAKqtFSHNHX99sy2

🎓 Exam Score Prediction – Full Machine Learning Pipeline

Author: ori manor
Assignment 2 – Regression, Classification, Clustering & Evaluation

A complete end-to-end ML project covering:
✔ EDA
✔ Research Questions
✔ Regression Models
✔ Feature Engineering
✔ Clustering
✔ Classification
✔ Model Selection & Deployment

---

📑 Table of Contents

1. Dataset Overview
2. Part 2 – Exploratory Data Analysis (EDA)
3. Part 2B – Research Questions
4. Part 3 – Regression Modeling
5. Part 4 – Feature Engineering
6. Part 5 – Improved Regression Models
7. Part 6 – Clustering
8. Part 7 – Regression → Classification
9. Part 8 – Classification Models
10. Final Model Selection
11. How to Use the Saved Models

---

<a name="dataset-overview"></a> 📘 Dataset Overview

The dataset contains 20,000 students described by academic, demographic, lifestyle, and behavioral indicators.

Main Features

Targets

• Regression: exam_score
• Classification: Low / Medium / High (quantile-based)

---

<a name="eda"></a> 🔍 Part 2 – Exploratory Data Analysis (EDA)

Below are representative visualizations demonstrating trends, distributions, correlations, and relationships.

---

📊 2.1 Exam Score Distribution

Insights

• Distribution is close to normal, centered around ~62
• Great baseline for regression models
• No extreme skew → stable modeling behavior

---

📘 2.2 Correlation Heatmap

Key Findings

✔ study_hours, class_attendance, and sleep_hours strongly correlate with performance
✔ No significant multicollinearity
✔ Behavioral variables dominate score prediction

---

📈 2.3 Study Hours vs Exam Score

Insight
Clear linear trend:
More study hours → higher score.
Diminishing returns appear after ~8–9 hours/day.

---

🧍 2.4 Gender Distribution

Balanced representation supports fair modeling.

---

<a name="research-questions"></a> 🔬 Part 2B – Research Questions

---

RQ1: Does class attendance impact score?

Insight
Strong positive effect.
Students with high attendance show significantly higher average scores.

---

RQ2: Does sleep quality matter?

Insight
Good sleepers outperform poor sleepers by ~10 points.
Sleep quality meaningfully affects academic output.

---

RQ3: Which study method performs best?

Ranking

1️⃣ Coaching
2️⃣ Mixed
3️⃣ Online
4️⃣ Self-study (lowest)

---

RQ4: Age vs Exam Score

Insight
Age is not a meaningful predictor.
Scores remain constant across age groups.

---

<a name="regression"></a> 📉 Part 3 – Regression Modeling

Baseline model: Simple Linear Regression.

---

3.1 Actual vs Predicted

Interpretation
Predictions align closely with the diagonal → strong model calibration.

---

3.2 Residual Distribution

Insight
Residuals approximate a normal distribution → assumptions of linear regression hold.

---

<a name="feature-engineering"></a> 🛠️ Part 4 – Feature Engineering

Includes:
✔ One-hot encoding
✔ Scaling
✔ PCA (2 components)
✔ KMeans clustering
✔ Ratios (sleep_study_ratio)

---

PCA + KMeans Visualization

Cluster meanings

• Cluster 0: low study / low attendance
• Cluster 1: mixed behavior
• Cluster 2: high-performance disciplined group

---

<a name="improved-regression"></a> 📈 Part 5 – Improved Regression Models

Models evaluated:

• Linear Regression (engineered)
• Random Forest
• Gradient Boosting

---

Regression Metrics Comparison

Summary Table

---

<a name="clustering"></a> 🌀 Part 6 – Clustering

Clustering adds nonlinear structure that improves tree-based models.

KMeans (k=3) used with PCA projections for insight into behavioral groupings.

---

<a name="classification-conversion"></a> 🔄 Part 7 – Regression → Classification

Exam scores converted into equal-sized quantile buckets:

• Low (bottom 33%)
• Medium (middle 33%)
• High (top 33%)

Classes remain balanced, supporting reliable evaluation.

---

<a name="classification"></a> 🧪 Part 8 – Classification Modeling

Models trained:

• Logistic Regression
• Random Forest
• Gradient Boosting

Performance mainly judged by recall, important for identifying struggling students.

---

Confusion Matrices

Logistic Regression

Random Forest

Gradient Boosting

---

Classification Insights

• Most misclassifications occur between Medium ↔ High
• Low scores are easiest to detect
• Logistic Regression performs best overall
  ✔ highest macro recall
  ✔ stable generalization
  ✔ interpretable

---

<a name="final-model-selection"></a> 🏆 Final Selected Models

Regression Winner

✔ Linear Regression
File: winning_model.pkl

Classification Winner

✔ Logistic Regression
File: winning_classification_model.pkl

Chosen for:

• Consistent performance
• Low overfitting
• High interpretability
• Best recall balance

---

<a name="use-model"></a> 💻 How to Use the Saved Model

import pickle
import numpy as np

with open("winning_classification_model.pkl", "rb") as f:
    model = pickle.load(f)

example = np.array([your_feature_vector]).reshape(1, -1)
prediction = model.predict(example)
print(prediction)

---

✅ End of README

Feel free to open an issue or reach out for improvements.