03Fraud · ML · MLOps
Real-time Fraud Detection.
A production-style fraud detection system combining cost-aware decision thresholds, model comparison, explainability, monitoring and deployment controls.
01 · Problem
Fraud models fail when accuracy is treated as the objective.
Highly imbalanced transaction data makes conventional accuracy misleading. The useful question is how much fraud can be captured at an acceptable operational and customer cost.
02 · Approach
Optimise the decision, not the leaderboard.
Logistic regression, random forest and XGBoost were compared using out-of-time validation and fraud-relevant metrics. A cost-based threshold was then layered on top of the champion model.
03 · Decisions
Designed for trade-offs, not demos.
AUPRC and KS were prioritised over raw accuracy. Threshold selection was separated from model training so risk appetite could change without rebuilding the model. Explainability and adverse-action style outputs were designed per transaction.
04 · Evidence
Strong discrimination with decision-aware gains.
The champion XGBoost model achieved a KS of 0.924 and AUPRC of 0.869 on the project dataset. Cost-based thresholding improved fraud capture by 11.2% relative to the baseline decision point.
05 · Reliability
Trust is part of the architecture.
The pipeline includes champion–challenger comparison, drift triggers, MLflow tracking, reproducible data/model versioning, API serving and a monitoring dashboard. Fairness checks are included as part of model governance.
06 · Reflection
Deployment changes what ‘best model’ means.
The project reinforced that a slightly better offline score is not automatically a better production choice. Latency, explainability, threshold economics and monitoring all influence the final system.
System architecture