joey_s_the_largest_diamond_dataset_price_prediction_models

<video controls width="100%"> <source src="Joseph_zeloof_Reg-Class_Model_Presentation.mp4" type="video/mp4"> Your browser does not support the video tag. </video>

Link to Youtube in case your browser dosnt support the video/it doesn't work/file too big to display:

https://youtu.be/h5M3qByIhrk

Part 1: Dataset Selection

1. Selected dataset: The largest diamond dataset currently on Kaggle

2. Link to dataset: https://www.kaggle.com/datasets/hrokrin/the-largest-diamond-dataset-currely-on-kaggle

3. Dataset description: This dataset contains the prices and other attributes of over 200k diamonds. Including many features about the diamonds quality, size, attributes and finnaly its price.

Part 2: Exploratory Data Analysis (EDA)

1. cleaning: the data was in pretty good shape, it had two categorical features that contained null which i change to categories labeled 'empty'. It also had 2 features that were percentegaes out of which i created two new features which binned those percentages in 10 sized intervals. The data also had lots of outliars so i decided to remove them using the IQR method

2. After that i created multiple plots showing some analysis of the data whereby the general insights where that diamonds are priced mainly in acorrdance to their to carrat weight. i also showed all plots of different features correlation to other features where i learned that most size features are positivly correleted to each other due to geology and market prefrences.

3. Example plot:

Part 3: Define and Train a baseline model

1. Regression Goal: To predict diamond price (total_sales_price) using available numeric and categorical features. this is very important as it will allow anyone to who holds this model to evaluate a stone in his possesion which is somthing diamond dealers multiple times a day manually.

2. Feature Selection: for the sake of simplicity i will includeonly all the aformentioned numeric columns and wont include the categorical ones - and we will continue from there

3. Results:

   • MAE: 396.13
   • MSE: 308,627.49
   • RMSE: 555.54
   • R²: 0.7905

• my r2 is suprisingly good given i havent included any of the categorical features which i thought would amount to a majority of the value but on the other hand my errors are in the hundreds of dollars which is pretty awful IMO given the fact that the avg price is around 7k$. nevertheless it is a good baseline to start from.

• Feature Coefficients:

  • carat_weight 4869.020428
  • meas_depth 536.917428
  • table_percent 8.743010
  • meas_length 5.660809
  • depth_percent -26.853394
  • meas_width -34.392333

1. Insights: in this innitial model we see that we already have a pretty good accuracy and we learn that the biggiest factors in determining the price of a diamond out of the numeric factors are the carrat and depth.

Part 4: Feature Engineering

1. I scaled all numeric features and encoded all the categorical ones using OneHotEncoder.
2. It should also be noted here that during part 2 of this assignment I dropped the id feature and added two binning features (one for tabl percent and one for depth percent)
3. Finnaly, i clustered all the rows into 5 clusters using KMeans and creted two new features based of them: cluster id and distance to its centroid. Then I analyzed the clusters ultimalty not gaining much helpful information from the clusters - an understanding that was strengthened by the fact the during later regression models the clusters werent the featurs with substantial coefficents.

Part 5: Training and Evaluating Three Improved Models

1. Here I utilized my new engineered features on three different models:
   1. normal regression
   2. random forrest regresssion
   3. gradiant boosting regression
2. In order to get the models to run i sampled 50k of the around 200k rows
3. Results: all models way outperfomed the baseline with the best one being the random forrest regressor with the following accuracy:
   1. MAE: 119.80
   2. RMSE: 217.77
   3. R²: 0.9675
4. the regression showed that above all else and by a hugemargin the thing that determines most of the value is the carrat.
5. 

Part 6: Winning Model

1. as mentioned the winning model was the random forrest so i exported it to .pkl file and uploaded it to HF

Part 7: Regression-to-Classification

1. I started by converting the my target label of price to two classes using a median split
2. I chose the median split method because it creates two balanced classes—“high-priced” and “low-priced”—which simplifies the classification task and avoids issues with class imbalance. Since diamond prices are heavily skewed and have a long tail, using the median instead of arbitrarily chosen thresholds produces a more stable and fair division of the target. This approach allows the classification models to focus on learning what differentiates relatively expensive diamonds from cheaper ones based on their features.
3. Then I deleted the two features that were binnings of other exsiting features as they wernt numeric and would interfere with the models
4. I checked the Balance and as expected the class distribution is balanced because the median split creates two groups of nearly equal size. No class is under-represented, so standard metrics such as accuracy, precision, recall, and F1 are all appropriate. No adjustments to the class conversion are needed.

Part 8: Train & Eval Classification Models

1. Precision vs Recall: In the context of predicting whether a diamond is high-priced, precision is generally more important than recall. Misclassifying a low-priced diamond as high-priced (a false positive) could mislead buyers, inflate expected revenue, or distort inventory decisions. High precision ensures that when the model predicts a diamond is expensive, it is very likely correct — which is crucial for pricing, sales strategy, and consumer trust. Recall is still useful, but missing some high-priced diamonds (false negatives) is less harmful than incorrectly labeling cheap diamonds as valuable.

2. false positives vs false negatives: A false positive is more critical as explained in the question above but to sum it up: because mislabeling a high-priced diamonds (false negatives) is less harmful than incorrectly labeling cheap diamonds as valuable mainly due to the pottential gain/loss invovlved.

3. I trained three models and displeyed classification reports and confusian matrixes for each one:

   1. Logistic Regression – linear baseline
   2. 
   3. Random Forest – tree-based ensemble
   4. 
   5. K-Nearest Neighbors (KNN) – instance-based, distance-based model
   6. 

4. surprisingly, in the classification models just like in the regression models the random forrest model had the highest accuracy acroos all stats. Classification Report: precision recall f1-score support

       0       0.96      0.96      0.96      4997
       1       0.96      0.96      0.96      5003
   

   accuracy 0.96 10000 macro avg 0.96 0.96 0.96 10000 weighted avg 0.96 0.96 0.96 10000

5. I choose it as the winner beacuse Random Forest achieves the highest overall performance, with the most balanced precision and recall across both classes and the lowest total number of misclassifications. Its ability to capture nonlinear patterns and interactions makes it better suited for the complex structure of diamond pricing than a linear model (Logistic Regression) or a distance-based model (KNN). While Logistic Regression is close, Random Forest still makes fewer errors overall and offers a more flexible and robust decision boundary.