AI for Loan Default Prediction: How It Works and Where It Breaks

 

Your credit model has never met the people you declined

Repayment is observed only for applicants who were approved. They passed your existing policy, so the training data describes the relationship between characteristics and default conditional on having already cleared a cutoff. The model is then applied to everyone who applies, including the population it has never seen.

This is the defining structural problem in credit modelling, and no choice of algorithm addresses it. It is also the one most often handled with a technique that sounds like a solution and is actually an assumption.

What selection actually breaks

Confidence without evidence. The model produces scores for applicants in regions of the feature space where it has no observations. Those scores are extrapolations, and nothing in the output distinguishes them from scores backed by thousands of outcomes.

Validation does not rescue you. The holdout comes from the same approved population, so strong validation confirms the model works on people you already lend to. It says nothing about applicants you do not serve, which is exactly where a policy change would take you.

Inherited judgement becomes fact. The previous policy encoded someone's view of who was creditworthy. The model learns it as an empirical finding, wrong parts included.

It compounds. Each model generation shapes the next set of approvals, which becomes the next training set. The population narrows quietly, and the metrics look fine throughout because they are computed on it.

The standard fixes are assumptions, not evidence

Reject inference techniques assign inferred outcomes to declined applicants through parcelling, augmentation or joint modelling schemes. They are not useless and they are not corrections.

Every one requires assuming something about rejected applicants that the data cannot establish, which is the original problem restated. Document the assumption, run sensitivity analysis across plausible alternatives, and report the range rather than a single adjusted figure. A performance claim resting on an untestable assumption should say so wherever it appears, including in the credit committee paper.

Unbiased information has a price

The only genuinely new evidence about the population you decline comes from approving some of them and observing what happens.

Two practical forms. A small randomised approval band below the cutoff produces unbiased outcomes for applicants just outside current policy, at a cost that can be estimated in advance. And swap set analysis, run whenever policy changes, tracks who is newly approved and newly declined under the new rules and how each performs, which is the closest most lenders get to a controlled comparison.

Neither is comfortable to propose. Both separate a model that can justify expanding the credit box from one that can only confirm it. Frame this as buying information rather than accepting losses, and get sign off at a level that can authorise the spend.

Probability is not the decision

Default probability is one of three components of expected loss. Loss given default depends on security, recovery process and time to recover. Exposure at default depends on product mechanics, particularly on revolving facilities where a distressed borrower draws down before failing.

A well secured loan with a higher default probability can be better business than an unsecured loan with a lower one, and ranking on default probability alone systematically misjudges that comparison.

Loss given default models are usually weaker than default models, because recoveries are lumpy, slow and sensitive to collateral markets. That weakness belongs in pricing and in the model inventory, not blended into a composite number where nobody can see which part is unreliable.

Pricing feeds back into the population

Risk based pricing creates an effect a default model does not capture. When you offer a higher rate to a higher risk applicant, the people most likely to accept are those without better options elsewhere. Acceptance therefore correlates with unobserved risk, and the accepted population at a given price is worse than the applicant population at that price.

Two consequences. Default rates after introducing risk based pricing are not comparable to those before, so a deterioration may be the pricing working as designed rather than a model failure. And portfolio outcomes depend on who is offered what, who accepts and who then repays, so an acceptance view belongs alongside the default model rather than after it.

Lenders who skip this usually find out through slow deterioration in the segments where they priced most aggressively.

Explainability and fair lending constrain model choice

In consumer lending an adverse decision commonly requires specific principal reasons. They must be accurate, stable across similar applicants and defensible, which constrains which models can be used at all rather than being a documentation step afterwards.

This is where the honest comparison sits. Monotonic scorecards, meaning logistic regression over binned characteristics with enforced direction, remain widely used for substantive reasons: reason codes fall out of the structure, monotonic constraints prevent counterintuitive behaviour such as penalising a longer credit history because of a data quirk, and monitoring is direct because a shift in one characteristic's contribution is visible. Gradient boosted models usually deliver better discrimination, typically a moderate rather than transformative gain. Post hoc explanation methods produce plausible reasons, but a reason derived from an approximation of the model is not one derived from its structure, and that distinction matters when a regulator asks how a decision was reached.

Fair lending belongs in model selection for the same reason. Removing protected characteristics is necessary and insufficient, because geography, device signals and name derived features act as proxies. In several jurisdictions the analysis extends to whether a less discriminatory alternative with comparable business results exists, which means searching during development and documenting what was considered rather than testing one candidate at sign off.

One monitoring item usually missed: track underwriter overrides by frequency, direction and subsequent performance. Persistent profitable overrides mean the model is missing something underwriters can see; persistent unprofitable ones are a governance problem. Most lenders do not capture either.

Where a blockchain does not help

Our core business is blockchain engineering, so this is worth saying directly. The recurring pitch in lending is portable credit history on a distributed ledger, or on chain scoring backed by decentralised identity.

It misreads the constraint twice. The barrier to portable credit data is bureau economics, governance and consent, not the absence of a shared database. And credit data must be correctable: consumers have statutory rights to dispute and amend inaccurate information in most jurisdictions with a credit reporting regime. Immutability works against that requirement, and any design has to add a correction layer that ends up being the real system of record.

What the deliverable includes

If someone builds this for you, the model file is the smallest part. The deliverable is the default definition and its documentation, the point in time feature pipelines, the reject inference assumptions with their sensitivity analyses, the fair lending alternatives considered and why they were not selected, and the validation artefacts.

The build to own principle applies here as it does to software. A credit model you cannot revalidate, re explain or defend without the original supplier is a dependency inside a regulated decision affecting consumers, and it will need defending sooner than anyone expects.

RWaltz is a blockchain and enterprise software development company building custom smart contracts, dApps and tokenization platforms that integrate with existing business systems. We work to a build to own model: clients hold their keys, repositories and intellectual property, engagements are scoped honestly including the cases where a simpler approach is the better answer, and review is treated as continuous rather than a single sign off.

📖 Read the full blog: https://www.rwaltz.com/blogs/ai-for-loan-default-prediction-how-it-works-and-where-it-breaks

Connect with RWaltz:

• LinkedIn: https://www.linkedin.com/company/rwaltzsoftware
• X (Twitter): https://twitter.com/rwaltzsoftware
• Facebook: https://www.facebook.com/RWaltz-Software-PvtLtd-255590135349493
• Telegram: https://t.me/RWaltzCrypto
• GitHub: https://github.com/rwaltzsoftware
• Clutch: https://clutch.co/profile/rwaltz-software
• Website: https://www.rwaltz.com