ieee-cis-fraud-detector

IEEE-CIS Fraud Detection

A lightgbm model that scores card transactions for fraud risk, trained on the IEEE-CIS Fraud Detection dataset.

Full project: https://github.com/Dee-ui/ieee-cis-fraud-detection

What is in this repository

Both files are needed. The model expects features in one exact order, which only the transformer produces.

How it performs

Measured on a held-out period that comes strictly after everything it was trained on, 2018-04-20 to 2018-05-31.

Operating threshold 0.4222, chosen by a cost model at a 2% manual review capacity, not left at the default 0.5.

Using it

import joblib, json, pandas as pd
from huggingface_hub import hf_hub_download

repo = "Dee-ui/ieee-cis-fraud-detector"
engineer = joblib.load(hf_hub_download(repo, "feature_engineer.joblib"))
model = joblib.load(hf_hub_download(repo, "final_model.joblib"))
metadata = json.load(open(hf_hub_download(repo, "final_model_metadata.json")))

# A raw transaction. Any column you leave out is treated as unknown.
transaction = pd.DataFrame([{
    "TransactionID": 3663549, "TransactionDT": 18403224,
    "TransactionAmt": 31.95, "ProductCD": "W", "card1": 10409,
}])

features = engineer.transform(transaction)
probability = model.predict_proba(features[metadata["feature_names"]])[:, 1][0]
print(probability, "review" if probability >= metadata["chosen_threshold"] else "pass")

Limitations worth knowing

• It catches about 44.6% of fraud by count but only 31.2% by value. Missed frauds average $186 against $105 for caught ones. Do not estimate savings by multiplying recall by total fraud losses.
• Roughly 10% of its decision weight sits on features derived from a customer fingerprint that is unavailable for about 82% of transactions in the later test period. Performance on data far from the training window should be monitored, not assumed.
• Trained on 2017 to 2018 data. Fraud patterns move.

Licence

MIT.