Loan default prediction.
Credit models tested on three things at once: performance, explainability and fairness. A missed default costs the bank; an unfair rejection costs the customer.
What the dissertation asked
Can a credit model be accurate, explain itself, and treat people fairly, or does chasing one of those quietly cost the others?
Four models were trained and compared, from a logistic regression baseline to gradient-boosted ensembles. Every prediction was then explained with SHAP at both global and individual level.
Rather than describing the results loosely, six research hypotheses were set up front and formally tested with statistical significance tests, including one about fairness that the model failed.
From raw data to a fairness verdict
- 01
Clean
Median and most-frequent imputation; outliers capped with Tukey’s fences at 3× IQR instead of dropped.
Winsorisation - 02
Engineer
Debt-to-income, income per employment year and default history. Age and income groups set aside before modelling.
24 features - 03
Train four models
Logistic regression, random forest, XGBoost and LightGBM, with class weights for the imbalance.
Stratified 70 / 30 - 04
Evaluate
ROC-AUC as the primary metric, plus precision, recall and F1, and 5-fold cross-validation on the training set only.
5-fold CV - 05
Explain
SHAP global importance, dependence plots and waterfall explanations for individual default and non-default cases.
SHAP - 06
Audit fairness
Demographic parity, equalised odds and disparate impact across age and income groups.
Post-hoc audit
Six claims, tested
- H1Accepted
Top SHAP features are significantly linked to default
4 of the 5 top features were significant (point-biserial and chi-square tests).
- H2Accepted
Boosted models beat the logistic regression baseline
LightGBM 0.932 and XGBoost 0.932 ROC-AUC against 0.881 for logistic regression.
- H3Accepted
Ensembles differ from each other
XGBoost and LightGBM were comparable, and both outperformed random forest.
- H4Accepted
SHAP agrees with each model’s native importance
SHAP rankings correlated significantly with native importance (Spearman), while adding per-prediction explanations.
- H5Rejected
The best model shows no significant demographic bias
Age passed every fairness metric. Income failed: demographic parity gap 0.189 and disparate impact 0.38.
- H6Accepted
Feature engineering improves performance
Accepted narrowly: only logistic regression improved significantly (+0.007 AUC). The tree models gained nothing measurable.
Four models, held-out test set
| Model | ROC-AUC | 5-fold CV | Precision | Recall | F1 |
|---|---|---|---|---|---|
| LightGBM · best | 0.932 | 0.926 ± 0.005 | 0.776 | 0.770 | 0.773 |
| XGBoost | 0.932 | 0.925 ± 0.005 | 0.793 | 0.770 | 0.782 |
| Random forest | 0.916 | 0.911 ± 0.005 | 0.750 | 0.746 | 0.748 |
| Logistic regression | 0.881 | 0.876 ± 0.005 | 0.558 | 0.789 | 0.653 |
How to read this. 22,806 applications for training and 9,775 held out, split with the default rate preserved. Cross-validation ran on the training set only, so the test set was never touched while choosing models. Low spread across folds (± 0.005) means the ranking is stable.
The strongest driver, in every model
SHAP ranked debt-to-income first across all four models, ahead of interest rate, income and loan-to-income share.
Feature engineering barely helped
Engineered features significantly improved only logistic regression. For the tree models the change was within noise, so the hypothesis passed on the narrowest possible terms, and I report it that way.
Accurate isn’t the same as fair
Age passed every test
- Demographic parity
- 0.015
- Equalised odds
- 0.017
- Disparate impact
- 0.93
Income failed all three
- Demographic parity
- 0.189
- Equalised odds
- 0.123
- Disparate impact
- 0.38
What I chose, and what I rejected
Report the failed hypothesisover quietly dropping the fairness result
H5 was rejected, and it is the most useful finding in the project: a model can top the accuracy table and still fail a fairness audit.
Set protected groups aside before modellingover leaving them in the feature set
Age and income groups were saved separately and dropped from the model, so they could be audited against without being predicted from.
Cap outliers at 3× IQRover 1.5× IQR or deleting rows
Income, loan size and interest rate are naturally skewed and can hold legitimate extremes. Winsorising kept every application while limiting their pull.
Test claims with statisticsover reading them off a chart
Each hypothesis was set before the results and decided by a named test: point-biserial, chi-square, Spearman and paired t-tests.
What this does and doesn’t prove
It does show
- Boosted models clearly beat a linear baseline on this data
- Explanations that agree with each model’s own importance
- A fairness audit that found a real income gap
- Every claim tied to a stated hypothesis and test
It doesn’t show
- Performance over time: the split is random, not out-of-time
- A fix for the income bias: it was measured, not mitigated
- Production behaviour: this is a research notebook, not a deployed system
- Real lending outcomes from one lender: it uses a public Kaggle dataset