pokemon_battle_Dor

🎥 Project Video

<video src="https://huggingface.co/RKugel/pokemon_battle_Dor/resolve/main/pokemon%20ML%20video.mp4" controls width="720"></video>

▶️ Watch the project walkthrough video

---

Pokémon Battle Outcome Predictor

Given two Pokémon, who wins? A complete data-science pipeline — from raw battle logs to a deployed model — built as part of the AI & Data Science course at Reichman University.

Author: Roie Kugel · Course: AI & Data Science (Assignment #2) · Date: April 2026

---

TL;DR

---

1. The Research Question

Can we predict, with high confidence, the outcome of a Pokémon battle from each combatant's base stats alone?

The project answers this question in two passes.

The regression pass estimates the probability that Player 1 wins — a continuous score between 0 and 1 that is interpretable as confidence. The classification pass turns that probability into a hard Win / Loss decision so the model can be deployed as a real predictor.

Both passes share the same feature pipeline. The split between regression and classification is methodological, not data-driven, and is explained in §7 and §8.

---

2. The Dataset

Source. Diarmuid27/pokemon-battle-outcomes on Hugging Face.

Size. 50,000 battles × 25 columns.

Schema.

The suffixes _First and _Second denote Player 1's and Player 2's Pokémon respectively.

2.1 What we cleaned, and why

After cleaning, every row in the dataset is a complete battle record.

---

3. The Story the Data Tells (EDA)

Five research questions guided the exploration. Each one is paired with a single chart and a one-paragraph answer.

3.1 Does Legendary status decide battles?

Answer — yes, decisively. When Player 1 fields a Legendary against a Common Pokémon, the win rate jumps to roughly 78%. The mirror scenario (Common vs. Legendary) collapses to 17%. Within the same tier (both Legendary, both Common) the win rate sits near 50% — a fair fight. Legendary status is the single most predictive feature in the raw dataset, and motivates the engineered legendary_advantage feature in §5.

---

3.2 Which stats correlate most strongly with winning?

Answer — Speed is the top single correlate, but no individual stat is enough on its own.

Note: total_first is a feature derived during EDA — it sums all six base stats of P1's Pokémon (hp_first + attack_first + defense_first + sp_atk_first + sp_def_first + speed_first). It is the standard "Base Stat Total" from the Pokémon games, used here as a single-number summary of overall power. total_second is the same sum for P2.

Reading the bottom row of the heatmap (correlation of each P1 feature with p1_won):

Speed alone explains the most variance in the outcome of any single feature — consistent with the §3.3 finding that whoever strikes first usually wins. total_first is third (after speed and attack). The heatmap also shows heavy multicollinearity between the offensive stats (attack ↔ sp_atk = 0.40, total ↔ each stat = 0.58–0.74), meaning a linear model fed all six stats would suffer from redundant signals.

This motivates engineering total_diff = total_first − total_second (the power gap between the two combatants) and total_ratio = total_first / total_second (the proportional advantage) — not because total is the strongest individual signal, but because aggregating the six correlated raw stats into one compact, multicollinearity-resistant feature gives the model a clean overall-power signal without six overlapping inputs.

---

3.3 Does being faster than your opponent translate into victory?

Answer — yes, with a clean separation. When P1 wins, the distribution of speed_first − speed_second skews positive. When P1 loses, it skews negative. The dashed line at zero (equal speed) sits cleanly between the two distributions. Speed advantage is the second-most reliable single signal after total stats.

---

3.4 Does the Pokémon's generation matter?

Answer — no, and showing this matters. Across every generation gap from −5 to +5, the win rate stays inside a narrow band of ≈ 0.45 to 0.55 — within the noise zone. The right-hand panel confirms sample sizes are large enough for the result to be trusted. Generation is dropped from the feature set. Including features that don't predict the target is a common student mistake; ruling them out explicitly is part of rigorous modeling.

---

3.5 Does total power gap separate winners from losers?

Answer — yes, this is the strongest engineered signal. The two KDE distributions barely overlap. P1 wins cluster on the positive side of zero (P1 had more total stats), losses cluster on the negative side. The vertical line at zero acts as a natural decision boundary — and indeed, an "always pick the higher-total Pokémon" heuristic would already achieve roughly 80% accuracy. The model's job is to push past that ceiling on the borderline matchups, where total stats are close.

---

4. Phase 1 — The Baseline (Linear Regression on Raw Features)

Before any feature engineering, we establish an honest reference point: Linear Regression trained on the raw per-Pokémon stats only — no gaps, no totals, no clusters. Just the 14 raw inputs the dataset hands us (12 stats + 2 legendary flags).

Result: R² ≈ 0.47, MAE ≈ 0.30.

That's decent — but the diagnostics show why it is a ceiling, not an answer.

The scatter forms two horizontal bands rather than tracking the diagonal — a tell-tale sign that a linear model is the wrong shape for a binary target. Residuals are bimodal, not normal.

Even at the baseline level, the model already detects the right signals: the legendary flags carry the strongest coefficients, followed by speed. This previews what later models will confirm at scale — but a linear model can never fully express the non-linear physics of a battle.

This baseline motivates everything that follows: feature engineering (§5), clustering (§6), and the three real models (§7).

---

5. Feature Engineering — Rebuilding the Feature Space

Raw per-Pokémon stats give the model 14 numeric columns. But the underlying physics of a battle is not "How big is P1's HP?" — it is "How does P1's HP compare to P2's?" Feature engineering rebuilds the dataset around relative signals.

5.1 The four families of engineered features

The engineered set replaces the 14 raw features with 15 richer, model-ready signals.

5.2 Visualizing the lift from engineering alone

Before introducing tree models, the same Linear Regression is retrained on the engineered features to isolate the lift produced by feature engineering by itself:

total_diff, legendary_advantage, and speed_gap rise to the top of the coefficient ranking — exactly the three signals EDA pre-flagged as most informative. This validates the engineering choices before the heavy artillery arrives.

---

6. Clustering — Discovering Battle Archetypes

A Pokémon battle is rarely "just stats." A glass-cannon vs. tank match plays differently than two balanced rivals. To capture this, we run K-Means clustering on the gap features and turn cluster identity into model inputs.

6.1 Choosing K

The elbow method on inertia from K = 2 to 10 lands cleanly at K = 4.

6.2 Visualizing the clusters with PCA

Reducing the 9-dimensional engineered feature space to two principal components (PC1 + PC2 ≈ 70% of variance) shows the four clusters as distinct regions of the space:

6.3 Three features derived from clustering

• cluster_id — which battle archetype does this match-up belong to?
• dist_to_centroid — how typical is this battle within its archetype? Small distance = textbook example. Large distance = ambiguous, on the boundary between archetypes.
• prob_cluster_k — soft membership scores derived from inverse distances. A battle split 0.45 / 0.40 / 0.10 / 0.05 is genuinely ambiguous; one at 0.95 / 0.02 / 0.02 / 0.01 is unambiguous. The model uses this to calibrate its own confidence.

---

7. Phase 2 — The Three Real Models

With the engineered + clustered features in hand, we move past linear models and train the three "real" candidates on the upgraded feature set:

For the two tree models, the probability output (.predict_proba()[:, 1]) plays the role of the continuous regression score. This avoids the structural weakness of Linear Regression on binary targets and gives a properly calibrated value in [0, 1].

🏆 7.1 The Winner: Gradient Boosting

After training and 5-fold cross-validation:

Gradient Boosting wins with 94.72% accuracy and AUC-ROC = 0.978. That is roughly 6 percentage points above Logistic Regression and a clear margin over Random Forest at every threshold. Cross-validation confirms the result is stable, not a lucky split.

The winning model is saved at pokemon_gradient_boosting.pkl. Call .predict_proba(X)[:, 1] on it to obtain the probability in [0, 1].

---

8. Reframing as Classification

The same problem can also be framed as classification: not "what is P(P1 wins)?" but "will P1 win — yes or no?" The target p1_won is already binary (1 = win, 0 = loss), so the engineered features carry over directly.

Class balance check. Classes are nearly balanced (≈ 52.7% losses, 47.3% wins, ratio 1.11 : 1). Accuracy is therefore a trustworthy headline metric.

The same three models — Logistic Regression, Random Forest, Gradient Boosting — are retrained on the engineered features in classification mode. Gradient Boosting wins again and is saved as pokemon_classification_model.pkl (a self-contained bundle: model + scaler + feature names).

---

9. Diagnostic Deep Dive

9.1 ROC curves — comparing the three models

The Gradient Boosting curve hugs the top-left corner more tightly than the others — it makes fewer false positives at every threshold, not just at the default 0.5.

9.2 Feature importance — what the models actually learned

Both tree models converge on the same top three drivers: total_diff, speed_gap, and legendary_advantage — the three signals that EDA pre-flagged as the most informative. The clustering features (cluster_id, dist_to_centroid, prob_cluster_*) contribute incrementally; they do not dominate, but they sharpen the borderline cases.

9.3 Confusion matrices — where the models still fail

The remaining errors concentrate on near-tie battles — match-ups where total_diff is close to zero and legendary_advantage is also zero. These are genuinely ambiguous battles where even a domain expert could not be certain of the outcome. The model is not failing on easy cases; it is failing on unfair-to-call cases.

---

10. Why Gradient Boosting Wins

1. Sequential error correction. Random Forest votes among independent trees. Gradient Boosting builds each new tree specifically to fix the previous trees' mistakes — exactly what the borderline near-tie cases need.
2. Native non-linearity. Battle outcomes depend on interaction terms (speed advantage matters most when total stats are close). Tree splits encode these interactions without being told to look for them; linear models cannot.
3. Direct loss optimization. Gradient Boosting optimizes log-loss at every step — the loss function aligned with the binary target.

The improvement over Random Forest is small (~0.2 pp) because the engineered features already do most of the work. This is the correct order of operations: good features first, fancy models second.

---

11. How to Use This Model

The repo ships two pickle files, each suited to a different use-case. Both are powered by the same Gradient Boosting model, but they are packaged differently and answer slightly different questions.

11.1 What each model returns

In short: the regression file gives you a percentage; the classification file gives you a decision (and the percentage if you ask for it). Both files describe the same underlying battle prediction — they just expose the answer at different levels of granularity.

11.2 Loading the regression model (probability output)

import pickle

with open("pokemon_gradient_boosting.pkl", "rb") as f:
    reg_model = pickle.load(f)

# X must already be scaled with the same StandardScaler used in training.
# Returns the probability of P1 winning (0.0 — 1.0).
probability = reg_model.predict_proba(X)[:, 1]
print(f"Chance of P1 winning: {probability[0]:.1%}")

11.3 Loading the classification model (decision + probability)

import pickle, pandas as pd

# This file is a SELF-CONTAINED bundle: model + scaler + feature names.
with open("pokemon_classification_model.pkl", "rb") as f:
    bundle = pickle.load(f)

model    = bundle["model"]      # GradientBoostingClassifier
scaler   = bundle["scaler"]     # fitted StandardScaler from training
features = bundle["features"]   # ordered list of expected feature names

# Build a one-row DataFrame containing the 15 engineered features
new_battle = pd.DataFrame([{
    "hp_gap": 30, "attack_gap": -15, "defense_gap": 10,
    "sp_atk_gap": 25, "sp_def_gap": -5, "speed_gap": 40,
    "total_diff": 85, "total_ratio": 1.18,
    "legendary_advantage": 1,
    "cluster_id": 2, "dist_to_centroid": 0.84,
    "prob_cluster_0": 0.05, "prob_cluster_1": 0.10,
    "prob_cluster_2": 0.78, "prob_cluster_3": 0.07,
}])

X = scaler.transform(new_battle[features])

decision    = model.predict(X)[0]                # 0 or 1
probability = model.predict_proba(X)[0, 1]       # P(P1 wins)

print(f"Outcome: {'P1 WINS' if decision == 1 else 'P1 LOSES'} "
      f"(confidence: {probability:.1%})")

11.4 Which file should I use?

Both files contain the same Gradient Boosting Classifier — the difference is only how it is packaged:

Either file supports both outputs:

For a yes/no answer, call .predict(). For a confidence score, call .predict_proba(X)[:, 1]. For a label that comes with its probability, call both.

---

12. Repository Contents

RKugel/pokemon_battle_Dor/
├── README.md                                       ← you are here
├── Rkugel_assigment_2_,_AI_data_science.ipynb      ← full notebook (EDA + modeling)
├── pokemon_gradient_boosting.pkl                   ← regression model (probability output)
├── pokemon_classification_model.pkl                ← classification model + scaler + features
├── 01_eda_legendary_scenarios.png
├── 02_eda_correlation_heatmap.png
├── 03_eda_speed_gap_violin.png
├── 04_eda_generation_gap_no_signal.png
├── 05_eda_total_stat_kde.png
├── 06_baseline_diagnostics.png
├── 07_baseline_coefficients.png
├── 08_kmeans_elbow.png
├── 09_pca_clusters.png
├── 10_engineered_coefficients.png
├── 11_roc_curves.png
├── 12_feature_importance.png
├── 13_confusion_matrices_part5.png
├── 14_class_balance.png
└── 15_confusion_matrices_classifiers.png

---

13. Citation

@misc{kugel2026pokemon,
  title  = {Pokémon Battle Outcome Predictor},
  author = {Kugel, Roie},
  year   = {2026},
  school = {Reichman University},
  course = {AI \& Data Science},
  note   = {Assignment \#2}
}

---

Acknowledgements

• Dataset: Diarmuid27/pokemon-battle-outcomes on Hugging Face.
• Toolchain: scikit-learn, pandas, numpy, matplotlib, seaborn.
• Course: AI & Data Science, Reichman University, Spring 2026.

---

If you read this far — thank you. Now go battle.

Dedicated to "Dor", my cousin's son. Thanks for the endless Pokémon stories and for giving me the best idea for this project.