weatherAUS-max-temp-regression-MaiG

Weather in Australia – Full ML Pipeline (Assignment 2)

Author: Mai Gurski
Course: Data Science / Machine Learning – Assignment 2
Dataset: Weather in Australia (daily observations, ~145K rows)

---

1. Project Overview

This notebook implements an end-to-end machine learning pipeline on the Weather in Australia dataset.

The goals are to:

1. Explore and understand the data through EDA and visualizations.
2. Engineer meaningful features (domain-based and data-driven).
3. Apply clustering to discover high-level weather patterns.
4. Build and compare regression models to predict the daily maximum temperature (MaxTemp).
5. Reframe the regression task as classification by creating temperature classes.
6. Train and evaluate three classification models and choose a winner.
7. Deploy the winning models to HuggingFace and document the process.

---

2. Links

• Notebook: Copy of Assignment #2 – Classification, Regression, Clustering, Evaluation.ipynb
• Winning regression model (pickle, Part 6):(https://huggingface.co/maigurski/weatherAUS-max-temp-regression-MaiG/blob/main/winning_model_part5.pkl)
• Winning classification model (pickle, Part 8.4): https://huggingface.co/maigurski/weatherAUS-max-temp-regression-MaiG/blob/main/best_classification_model_p8.pkl
• Presentation video (4–6 minutes): https://youtu.be/AfDMoXZgdQk

---

3. Dataset

The dataset contains daily weather observations from multiple locations in Australia.

• Rows: ~145,000 days
• Core features (examples):
  • Temperatures: MinTemp, MaxTemp, Temp9am, Temp3pm
  • Humidity: Humidity9am, Humidity3pm
  • Wind: WindSpeed9am, WindSpeed3pm, WindGustSpeed, WindDir9am, WindDir3pm, WindGustDir
  • Pressure: Pressure9am, Pressure3pm
  • Cloud cover, RainToday, RainTomorrow, Location, Date

In this notebook I focus on:

• Classification task:
  Predicting a 3-class target MaxTemp_class (cool / medium / hot days).
• Regression task:
  Predicting the continuous target MaxTemp.

---

4. Notebook Structure

1. Part 1 – Data Loading & Cleaning

   • Load the CSV, parse dates, handle missing values, and basic type conversions.

2. Part 2 – Exploratory Data Analysis (EDA)

   • Summary statistics, distributions, correlations and key visualizations.

3. Part 3 – Baseline Models

   • Initial regression and classification baselines using raw features.

4. Part 4 – Feature Engineering & Clustering

   • Create engineered features.
   • Apply K-Means clustering and visualize clusters using PCA.

5. Part 5 – Regression with Engineered Features

   • Train Linear Regression, Random Forest and Gradient Boosting on the engineered dataset.

6. Part 6 – Winning Regression Model

   • Select the best regression model and export it as a pickle file.

7. Part 7 – Regression-to-Classification

   • Create temperature classes from the numeric target using quantile binning.
   • Check class balance and discuss metrics.

8. Part 8 – Train & Evaluate Classification Models

   • Train Logistic Regression, Random Forest and Gradient Boosting classifiers.
   • Evaluate using classification reports, confusion matrices and macro F1.
   • Choose and save the winning classifier.

9. Part 9 – Presentation Video

   • Link to the recorded walkthrough of the project.

---

5. Exploratory Data Analysis (EDA) & Research Questions

In Part 2 I explored the Weather in Australia dataset to understand data quality, basic patterns and relationships before building any models.

2.1 Data cleaning and structure

• Parsed the Date column to a proper datetime type and kept only relevant columns.
• Checked for duplicates and removed them if present.
• Inspected data types and fixed inconsistencies (for example, categorical vs. numeric fields).
• Handled missing values:
  • For the regression and classification targets (MaxTemp, RainTomorrow, MaxTemp_class) I dropped rows with a missing target.
  • For numeric features I used simple strategies such as median imputation or row dropping, depending on how critical the feature was.
  • For categorical features I grouped rare categories and, when needed, used an "Unknown" category for missing values.
• Verified that the final working dataset had no NA values in the features used for modeling.

2.2 Outlier detection and handling

• Inspected distributions and box plots for key numeric features such as Rainfall, MaxTemp, MinTemp, Humidity3pm and wind speed.
• Identified extreme outliers in Rainfall (values up to 371 mm).
• Applied a mild winsorization to Rainfall at the 99th percentile (clipping very extreme days), without removing any rows:
  • Original max ≈ 371.0
  • Clipped max at 99th percentile ≈ 37.2
• All other numeric variables showed reasonable behaviour, so no additional outlier removal was applied.

2.3 Descriptive statistics

• Computed descriptive statistics (mean, median, std, min, max, quartiles) for all numeric variables.
• Examined the correlation matrix between weather features and the target MaxTemp.
• Observed strong positive correlations between MaxTemp and:
  • MinTemp
  • Temp3pm and Temp9am
• Also found meaningful relationships between humidity, rainfall and the rain indicator RainTomorrow, which later guided feature engineering.

2.4 Exploratory visualizations

To better understand the data, I used several visualizations:

• Histograms and KDE plots for key numeric variables (temperatures, rainfall, humidity, pressure).

• Box plots of MaxTemp by Location and by Season.

• Scatter plots (e.g., Rainfall vs MaxTemp, Humidity3pm vs MaxTemp).

• A correlation heatmap to summarize linear relationships.

• PCA plots later used to visualize clusters (see Part 4).

These plots allowed me to identify skewed distributions, potential non-linear relationships and variables that are likely to be important for prediction.

2.5 Research questions (6) and visual answers

Based on the EDA, I defined six research questions and answered each of them with specific plots and short written interpretations:

1. RQ1 – MaxTemp by location and season
   How does the distribution of daily maximum temperature (MaxTemp) differ across locations and seasons in Australia?
   → Answered with box plots of MaxTemp by Location and by Season.

2. RQ2 – MaxTemp vs. rainfall
   What is the relationship between daily rainfall and MaxTemp? → Answered with scatter plots and trend patterns showing how heavy rainfall days tend to be cooler.

3. RQ3 – Seasonality of temperature
   How does MaxTemp vary across the four seasons (summer, autumn, winter, spring)? → Answered with seasonal box plots highlighting higher temperatures and larger variability in summer vs. winter.

4. RQ4 – Humidity and rain tomorrow
   How are afternoon humidity levels (Humidity3pm) related to the likelihood of rain the next day (RainTomorrow)?

   → Answered with density/box plots comparing humidity distributions for RainTomorrow = Yes vs. No.

5. RQ5 – Intraday temperature dynamics
   How do intraday temperature metrics (such as TempRange and TempChange_9am_3pm) relate to cloud cover, wind and other conditions? → Answered with scatter plots and correlation views using the engineered features.

6. RQ6 – Weather “types”
   Can we discover meaningful weather “types” using clustering, and do they help explain variation in MaxTemp and humidity? → Answered by applying K-Means, visualizing clusters in PCA space and summarizing each cluster’s average profile (hot & dry, cool & humid, etc.).

These questions and visual answers bridge the gap between raw EDA and the modeling parts of the project: they directly motivated the engineered features, the clustering step, and the choice of models and metrics later on.

6. Feature Engineering

To improve model performance and capture domain structure, I engineered several new features:

• Temperature-based features

  • TempRange = MaxTemp - MinTemp
  • TempChange_9am_3pm = Temp3pm - Temp9am

• Humidity & pressure changes

  • HumidityDiff_9am_3pm = Humidity3pm - Humidity9am
  • PressureDiff_9am_3pm = Pressure3pm - Pressure9am

• Seasonality

  • Season (Winter / Spring / Summer / Autumn), derived from the date.
  • Month_sin, Month_cos – cyclical encoding of the month to capture yearly periodicity.

• Clustering feature

  • weather_cluster – cluster ID (0–4) from K-Means on standardized weather variables.

• Classification targets

  • MaxTemp_bin / MaxTemp_class – 3-class label for daily maximum temperature, based on quantile thresholds (approximately 33% / 66%).

These engineered features are used consistently across both regression and classification pipelines.

---

7. Clustering (Part 4.2)

I applied K-Means on standardized weather features (MinTemp, Temp3pm, humidity, pressure, wind, etc.):

• Number of clusters: k = 5
• Visualization: PCA projection with cluster colors.
  

Interpretation:

• Some clusters correspond to cooler and more humid days, others to hot and dry days.
• The cluster profiles table (mean values per cluster) shows distinct combinations of temperature, humidity and wind.
• The categorical feature weather_cluster was added to the dataset and later turned out to be important for the models.

---

8. Regression Models (Part 5)

Target: MaxTemp (continuous).
Training is done on the feature-engineered dataset using a consistent preprocessing pipeline:

• Numeric features: StandardScaler
• Categorical features: OneHotEncoder (Location, wind directions, RainToday, Season, etc.)

8.1 Models

1. Linear Regression + FE
2. Random Forest Regressor + FE (trained on a subsample of the train set for runtime)
3. Gradient Boosting Regressor + FE (also on a subsample)

8.2 Results (test set)

8.3 Interpretation

• All models achieve very strong performance, explaining most of the variance in MaxTemp.
• Gradient Boosting + FE performs best with the highest R² and lowest error.
• There is no severe overfitting: train and test metrics are close for all models.

8.4 Feature Importance (Regression)

Using the Gradient Boosting model:

• Top features include:
  • Temp3pm, Temp9am, TempRange, TempChange_9am_3pm
  • weather_cluster
  • specific locations (e.g., Location_Katherine)
  • humidity and pressure differences (HumidityDiff_9am_3pm, Pressure3pm)

These results confirm that both raw temperature measurements and engineered daily-change features are crucial for predicting maximum temperature.

---

9. Regression-to-Classification (Part 7)

To create a discrete target for classification:

• I used quantile binning on the train set only:
  • q1 (33%) ≈ 19.4°C, q2 (66%) ≈ 26.0°C.
  • Class 0 – cool days (below q1)
  • Class 1 – medium days (between q1 and q2)
  • Class 2 – hot days (above q2)

9.1 Class Balance

Class proportions are almost perfectly balanced:

• Train: ~0.325 / 0.337 / 0.338
• Test: ~0.325 / 0.336 / 0.339

Because the classes are balanced, accuracy is meaningful, but I still focus on macro F1 to give each class equal weight.

---

10. Classification Models (Part 8)

Using the same engineered features and preprocessing as in regression, I trained three classifiers for MaxTemp_class:

1. Multinomial Logistic Regression
2. Random Forest Classifier
3. Gradient Boosting Classifier

10.1 Metrics (test set)

10.2 Confusion Matrices

For each model I plot the confusion matrix on the test set.

Observations:

• All models correctly classify the majority of days in each class.
• Most errors occur between medium and hot classes, which is expected when temperatures are near the thresholds.
• Gradient Boosting provides the best trade-off across all metrics and is chosen as the winning classifier.

---

11. Winning Models & HuggingFace (Parts 6 & 8.4)

• Winning regression model: Gradient Boosting Regressor with engineered features (Part 5).

  • Saved as: models/best_regression_model_p6.pkl.
  • Uploaded to HuggingFace model repository.

• Winning classification model: Gradient Boosting Classifier with engineered features (Part 8).

  • Saved as: models/best_classification_model_p8.pkl.
  • Uploaded to the same / another HuggingFace repository.

Both models can be loaded via pickle and used to predict MaxTemp (regression) or MaxTemp_class (classification) on new data with the same preprocessing steps.