Building a Credit Scoring Model From Scratch
The Home Credit Default Risk dataset is 307,511 loan applications spread across 8 CSV files. There is a main application table and five auxiliary tables covering bureau credit history, previous applications, installment payments, POS cash balance, and credit card balance. The interesting modeling problem is not in the main table. It is in figuring out what to do with all the auxiliary data.
Feature engineering is where the work actually is
Each auxiliary table gets its own set of aggregations. Bureau history becomes 13 features: active credit ratio, max overdue days, average credit duration. Previous applications become 10 features: approval rate, average credit requested. Installments become payment ratio and average days late. By the time everything is joined and domain ratios are added, the feature matrix is 176 columns.
The single most important feature, by a wide margin, is ext_source_prod23 — the product of EXT_SOURCE_2 and EXT_SOURCE_3. These are external credit scores from two different bureaus. Multiplying them together captures an interaction that neither score captures alone: an applicant who scores poorly on both external sources is a much higher risk than one who scores poorly on just one. That interaction sits at the top of the global SHAP importance chart by a significant gap.
The class imbalance problem
Only 8.1% of applications in this dataset are actual defaults. If you train a naive model it will predict "no default" on almost everything and still get 92% accuracy. That accuracy number is meaningless. The real measure is ROC-AUC, which ignores the class distribution.
I handled the imbalance by setting scale_pos_weight to 11.3 (the ratio of non-defaults to defaults) and running Optuna hyperparameter optimization for 30 trials tracked in MLflow. The final model hit a validation ROC-AUC of 0.7743 and a test ROC-AUC of 0.7795. Those numbers are stable across the train/val/test split, which matters more than the absolute value.
Explainability without the overhead
In real lending, you cannot just say "denied." Regulation requires an explanation. SHAP gives you that explanation at the individual prediction level, per application, in the same API call.
I used XGBoost native TreeSHAP instead of the shap library. XGBoost exposes pred_contribs=True directly, which runs the same SHAP algorithm but avoids the shap/llvmlite dependency entirely. The result is SHAP values per feature per prediction with zero additional inference overhead. A declined application comes back with a list of the top drivers: ext_source_2 decreased risk by X, days_birth increased it by Y. That is an adverse action explanation.
How it runs in production
The API is FastAPI with Pydantic v2 validation. A POST to /score accepts whatever fields are available (the model handles missing values through median imputation), runs the sklearn preprocessing pipeline, scores with XGBoost, computes SHAP drivers, and returns a default probability, risk band (LOW / MEDIUM / HIGH), and the top feature contributions in a single response. Warm latency is under 35ms. The whole thing runs in a Docker container on Google Cloud Run. There is also a Streamlit dashboard that calls the live API and visualises the probability gauge and SHAP waterfall chart interactively.
The test suite has 41 tests across the pipeline, model training, SHAP, and API layers. The thing I kept coming back to on this project: the hard parts are not the model. They are the preprocessing pipeline that has to be identical between training and inference, the class imbalance that makes every accuracy metric misleading, and getting the explanation out of the model in a way that is actually useful under a latency constraint.