flight-price-random-forest

Flight Price Prediction — Regression, Classification & Clustering

Author: Shira Bash

project walkthrough presentation: Loom Video

---

Project Overview

This project aims to analyze and predict flight prices using multiple machine learning approaches.
The assignment includes:

• Exploratory Data Analysis (EDA)
• Regression modeling (baseline → advanced)
• Feature Engineering
• Clustering (K-Means + PCA visualization)
• Transforming regression into classification
• Training & evaluating classification models

Project Goal

The goal is to understand what drives flight prices, build effective predictive models, and communicate insights clearly.

---

Part 1 - Dataset and Overview

The dataset contains detailed information about commercial flights, including airline, departure/arrival times, number of stops, duration, and price. After cleaning (removing duplicates, dropping an unnamed column, and removing outliers in two columns using IQR), the final dataset shape is:

• 298,128 rows
• 24 columns

Main columns used:

• airline, source_city, departure_time, stops, arrival_time

• destination_city, class, duration, days_left

• Target (Regression): price

---

Part 2 - EDA (Exploratory Data Analysis)

Goals:

• Understand price distribution
• Explore relationships between price and flight features
• Identify key factors influencing price

Key steps performed:

• Visualizing distributions (price, duration, stops, class)
• Checking correlations
• Detecting and removing outliers

Placeholders for EDA Graphs:

Price Distribution

Conclusion: Ticket prices are highly right-skewed: most tickets are relatively cheap, with a small number of very expensive flights forming a long right tail of the distribution.

Price by Stops

Conclusion: Flights with one or more stops tend to be much more expensive than non-stop flights, and 1-stop itineraries are the priciest on average – showing that the number of stops is strongly related to ticket price.

Price by Duration

Conclusion: Longer flights generally have higher ticket prices, but the scatter is large, so duration alone cannot explain price; other features such as class, airline and number of stops also play an important role.

---

Research Questions

1. How does the average ticket price vary by month? Conclusion: The average ticket price drops noticeably from February to March. This suggests a seasonal pattern where flights in March tend to be cheaper, possibly due to lower demand or post-holiday travel trends.

2. How do ticket prices vary across travel class and number of stops? Conclusion: Business-class tickets are consistently more expensive than Economy across all stop categories. Prices also increase sharply with additional stops, especially for Business class. This indicates a strong interaction between travel class and number of stops in determining ticket price.

3. What is the relationship between flight duration and ticket price for non-stop flights? Conclusion: There is a positive trend—longer non-stop flights tend to have higher ticket prices. However, the large scatter shows that duration alone cannot fully explain price variation, meaning additional factors like class, route, and airline also significantly influence ticket cost.

4. How is the price distribution different between Economy and Business class? Conclusion: Economy tickets are heavily concentrated in the low-price range, while Business-class tickets appear at much higher prices with a wider spread. This clear separation shows that travel class is one of the strongest predictors of ticket price.

These insights were later connected to model performance and feature importance.

---

Part 3 - Regression Models

Baseline Model — Linear Regression

Using cleaned + encoded baseline features (11 features).

Results:

Train set:

• MAE ≈ 4,493
• RMSE ≈ 6,880
• R² ≈ 0.91

Test set:

• MAE ≈ 4,470
• RMSE ≈ 6,880
• R² ≈ 0.91

The baseline model performs strongly, showing that the engineered features are meaningful.

Baseline Regression Plot

Conclusion: The baseline linear regression captures the general upward trend between true and predicted prices, but many points deviate from the ideal diagonal line. This indicates that the model struggles with non-linear patterns and underfits in several price ranges, especially for very cheap and very expensive tickets.

---

Part 4 - Feature Engineering

To improve model performance, I added several engineered features on top of the baseline numeric features.

Added Features

• duration_hours – flight duration converted from minutes to hours
• is_long_flight – indicator for long flights (duration ≥ 600 minutes)
• is_non_stop – indicator for non-stop flights (stops == 0)
• is_night_arrival – indicator for night arrivals

K-Means Features

Using K-Means (k = 4), I created:

• cluster_id – assigned cluster label

• cluster_distance – distance to cluster centroid

• PCA 2D visualization of clusters

These features help the models capture non-linear patterns and interactions in the data.

---

Part 5 - Improved Regression Models + Part 6 - Winning Model

Models:

1. Random Forest Regressor
2. Gradient Boosting Regressor

Results summary:

Random Forest was the winning regression model.

Feature Importance Plot

Conclusion: The Random Forest model identifies class as by far the most important feature for predicting ticket prices. Other features such as duration_in_min, route_index, and airline_index contribute much less but still show meaningful influence. This confirms that travel class is the strongest driver of ticket pricing in the dataset.

---

Part 7 - Regression-to-Classification

In addition to predicting the exact ticket price, I also reframed the problem as a 3-class classification task.
The goal here is to classify each flight into a price level (cheap / medium / expensive) based on the numeric target price.

Creating the price_class target

To create the classification labels I transformed the continuous price variable into three classes using quantile binning on the training set only:

• Class 0 – cheap tickets – bottom 33% of prices
• Class 1 – medium tickets – middle 33%
• Class 2 – expensive tickets – top 33%

This strategy has two advantages:

1. It matches an intuitive business interpretation of cheap / medium / expensive tickets.
2. Using quantiles keeps the classes relatively balanced, so the classifier does not focus on one dominant class.

The resulting discrete labels are stored in a new target column called price_class, which is used throughout the classification part.

Class balance

After creating the labels, I checked the class distribution in both the train and test sets.
Each of the three classes (0 – cheap, 1 – medium, 2 – expensive) contains roughly one third of the samples in both splits (≈ 32–34% per class).
This means the classes are reasonably balanced and:

• Overall Accuracy is a meaningful metric.
• I also track Macro F1 score to make sure the model performs well across all classes, not just one.

Part 8 - Classification Models

Using the engineered features from the regression part, I trained several classification models to predict the new target price_class (0 = cheap, 1 = medium, 2 = expensive).

Business perspective: Precision vs Recall

From a user perspective, the most important group is the cheap tickets (class 0):

• I prefer the model to catch as many truly cheap flights as possible, even if some non-cheap flights are mistakenly labeled as cheap.
• Therefore, for the cheap class, Recall is more important than Precision.

In terms of error types:

• A False Negative (a truly cheap flight classified as not cheap) is more critical than a False Positive (a not-cheap flight classified as cheap), because the user will never see a good deal that actually exists.

Trained classification models

I trained three different classifiers on the same feature matrix (X_train_fe, X_test_fe) and the new target price_class:

1. Logistic Regression (multiclass baseline) – a simple linear baseline for multi-class classification.
2. Random Forest Classifier – an ensemble of decision trees that can capture non-linear relationships.
3. Gradient Boosting Classifier – another tree-based ensemble that focuses on correcting previous errors and often gives strong performance.

All models were evaluated on the same train/test split using:

• classification_report (precision, recall, F1, support) for train and test sets
• Confusion matrix on the test set
• Overall Accuracy and Macro F1 on the test set

Evaluation of classification models

On the test set, the models achieved the following results:

Because the classes are balanced, these metrics can be compared fairly across models.

Types of mistakes (based on confusion matrices)

• Logistic Regression

• 

  • Makes many mistakes between class 0 (cheap) and class 1 (medium).
  • Also confuses class 2 (expensive) with class 1, so expensive tickets are often predicted as medium.
  • The confusion matrix is more “spread out”, which matches the lower accuracy/F1.

• Gradient Boosting

• 

  • Performs better than Logistic Regression but worse than Random Forest.
  • Still shows noticeable confusion between class 1 and 2.
  • The diagonal in the confusion matrix is strong, but off-diagonal cells are larger than in Random Forest.

• Random Forest

• 

  • The confusion matrix is strongly concentrated on the diagonal.
  • Only a small number of tickets are misclassified, mostly between neighbouring classes (0 vs 1, 1 vs 2).
  • Very few cheap tickets are predicted as expensive (0 → 2) or the opposite (2 → 0).
  • This model captures the non-linear relationships in the data much better than Logistic Regression.

Winning classifier

The Random Forest Classifier is the winning model for the classification task because:

• It has the highest Accuracy and Macro F1 on the test set.
• It makes relatively few “severe” mistakes between cheap and expensive tickets.

---

Files Included

• Loom Video - project walkthrough presentation

• Freshly_cleaned.csv - dataset used for the analysis

• Assignment#2-Classification,Regression,Clustering,Evaluation-Shira Bash.ipynb – full Python notebook

• README.md – summary of results, questions, and insights

• flight_price_random_forest_classifier.pkl - 1 pickle model for classification

• flight_price_random_forest.pkl - 1 pickle model for regression

---

Final Notes

This project demonstrates the full ML lifecycle:

• Data understanding
• Cleaning & preprocessing
• Feature engineering
• Regression
• Classification
• Clustering
• Model evaluation
• Production-ready file export