A Quick Guide to Model Explainability

by Julie Lee 6 min read January 11, 2024

Model explainability has become a hot topic as lenders look for ways to use artificial intelligence (AI) to improve their decision-making. Within credit decisioning, machine learning (ML) models can often outperform traditional models at predicting credit risk.

ML models can also be helpful throughout the customer lifecycle, from marketing and fraud detection to collections optimization. However, without explainability, using ML models may result in unethical and illegal business practices.

What is model explainability? 

Broadly defined, model explainability is the ability to understand and explain a model’s outputs at either a high level (global explainability) or for a specific output (local explainability).1

  • Local vs global explanation: Global explanations attempt to explain the main factors that determine a model’s outputs, such as what causes a credit score to rise or fall. Local explanations attempt to explain specific outputs, such as what leads to a consumer’s credit score being 688. But it’s not an either-or decision — you may need to explain both.

Model explainability can also have varying definitions depending on who asks you to explain a model and how detailed of a definition they require. For example, a model developer may require a different explanation than a regulator.

Model explainability vs interpretability

Some people use model explainability and interpretability interchangeably. But when the two terms are distinguished, model interpretability may refer to how easily a person can understand and explain a model’s decisions.2 We might call a model interpretable if a person can clearly understand:

  • The features or inputs that the model uses to make a decision.
  • The relative importance of the features in determining the outputs.
  • What conditions can lead to specific outputs.

Both explainability and interpretability are important, especially for credit risk models used in credit underwriting. However, we will use model explainability as an overarching term that encompasses an explanation of a model’s outputs and interpretability of its internal workings below.

ML models highlight the need for explainability in finance

Lenders have used credit risk models for decades. Many of these models have a clear set of rules and limited inputs, and they might be described as self-explanatory. These include traditional linear and logistic regression models, scorecards and small decision trees.3

AI analytics solutions, such as ML-powered credit models, have been shown to better predict credit risk. And most financial institutions are increasing their budgets for advanced analytics solutions and see their implementation as a top priority.4 

However, ML models can be more complex than traditional models and they introduce the potential of a “black box.” In short, even if someone knows what goes into and comes out of the model, it’s difficult to explain what’s happening without an in-depth analysis.

Lenders now have to navigate a necessary trade-off. ML-powered models may be more predictive, but regulatory requirements and fair lending goals require lenders to use explainable models.

READ MORE: Explainability: ML and AI in credit decisioning

Why is model explainability required?

Model explainability is necessary for several reasons:

  • To comply with regulatory requirements: Decisions made using ML models need to comply with lending and credit-related, including the Fair Credit Reporting Act (FCRA) and Equal Credit Opportunity Act (ECOA). Lenders may also need to ensure their ML-driven models comply with newer AI-focused regulations, such as the AI Bill of Rights in the U.S. and the E.U. AI Act.
  • To improve long-term credit risk management: Model developers and risk managers may want to understand why decisions are being made to audit, manage and recalibrate models. 
  • To avoid bias: Model explainability is important for ensuring that lenders aren’t discriminating against groups of consumers.
  • To build trust: Lenders also want to be able to explain to consumers why a decision was made, which is only possible if they understand how the model comes to its conclusions.

There’s a real potential for growth if you can create and deploy explainable ML models. In addition to offering a more predictive output, ML models can incorporate alternative credit data* (also known as expanded FCRA-regulated data) and score more consumers than traditional risk models. As a result, the explainable ML models could increase financial inclusion and allow you to expand your lending universe.

READ MORE: Raising the AI Bar

How can you implement ML model explainability?

Navigating the trade-off and worries about explainability can keep financial institutions from deploying ML models. As of early 2023, only 14 percent of banks and 19 percent of credit unions have deployed ML models. Over a third (35 percent) list explainability of machine learning models as one of the main barriers to adopting ML.5 

Although a cautious approach is understandable and advisable, there are various ways to tackle the explainability problem. One major differentiator is whether you build explainability into the model or try to explain it post hoc—after it’s trained.

Using post hoc explainability

Complex ML models are, by their nature, not self-explanatory. However, several post hoc explainability techniques are model agnostic (they don’t depend on the model being analyzed) and they don’t require model developers to add specific constraints during training.

Shapley Additive Explanations (SHAP) is one used approach. It can help you understand the average marginal contribution features to an output. For instance, how much each feature (input) affected the resulting credit score.

The analysis can be time-consuming and expensive, but it works with black box models even if you only know the inputs and outputs. You can also use the Shapley values for local explanations, and then extrapolate the results for a global explanation.

Other post hoc approaches also might help shine a light into a black box model, including partial dependence plots and local interpretable model-agnostic explanations (LIME).

READ MORE: Getting AI-driven decisioning right in financial services 

Build explainability into model development

Post hoc explainability techniques have limitations and might not be sufficient to address some regulators’ explainability and transparency concerns.6 Alternatively, you can try to build explainability into your models. Although you might give up some predictive power, the approach can be a safer option. 

For instance, you can identify features that could potentially lead to biased outcomes and limit their influence on the model. You can also compare the explainability of various ML-based models to see which may be more or less inherently explainable. For example, gradient boosting machines (GBMs) may be preferable to neural networks for this reason.7

You can also use ML to blend traditional and alternative credit data, which may provide a significant lift — around 60 to 70 percent compared to traditional scorecards — while maintaining explainability.8

READ MORE: Journey of an ML Model 

How Experian can help

As a leader in machine learning and analytics, Experian partners with financial institutions to create, test, validate, deploy and monitor ML-driven models. Learn how you can build explainable ML-powered models using credit bureau, alternative credit, third-party and proprietary data. And monitor all your ML models with a web-based platform that helps you track performance, improve drift and prepare for compliance and audit requests.

*When we refer to “Alternative Credit Data,” this refers to the use of alternative data and its appropriate use in consumer credit lending decisions, as regulated by the Fair Credit Reporting Act. Hence, the term “Expanded FCRA Data” may also apply and can be used interchangeably.

1-3. FinRegLab (2021). The Use of Machine Learning for Credit Underwriting

4. Experian (2022). Explainability: ML and AI in credit decisioning

5. Experian (2023). Finding the Lending Diamonds in the Rough

6. FinRegLab (2021). The Use of Machine Learning for Credit Underwriting

7. Experian (2022). Explainability: ML and AI in credit decisioning

8. Experian (2023). Raising the AI Bar

Related Posts

How GenAI Fraud Bots Fly Under the Radar (and How to Ground Them Before It’s Too Late)

Today’s AI-powered fraud bots are exceptionally good at mimicking human behavior. Learn why a modern defense strategy is key.

November 18, 2026 by Laura Burrows
What Is Incident Response? Key Steps, Planning and Best Practices

Learn what incident response is, the key steps in the process and how to build and test an incident response plan for a data breach.

November 17, 2026 by Laura Burrows
Rethinking Charged-Off Portfolio Prioritization

Experian’s new Recovery Scores model turns portfolio intelligence into more focused recovery strategies, helping organizations improve returns, direct effort and accelerate speed to cash. The challenge isn’t simply working more accounts. Whether a debt buyer is evaluating an acquisition, an agency is deciding where to assign collector time or a lender is managing its own post-charge-off accounts, the question is similar: How can limited capacity be directed toward accounts with the greatest potential for recovery? Prioritizing accounts mainly by age, balance or when they were last worked may not show which ones are most likely to generate a recovery. With the new Recovery Scores model, organizations can rank charged-off accounts by expected collectability, providing a consistent signal to inform placement strategy and treatment decisions across the recovery lifecycle. Make collector capacity a strategic resource Working every account with the same level of attention isn’t always practical. The more useful question is: Where should teams focus first to protect speed to cash? Our model performance analysis found: 80% of recovered dollars come from the top 40% of accounts worked. The value isn’t simply a better ranking. It’s how organizations can use it for collections optimization. Score bands can inform different strategies for placement, outsourcing, auction and legal channel treatment rather than applying the same approach across an entire portfolio. For debt buyers: The scores can inform acquisition pricing and how a newly purchased portfolio is divided across internal collections, legal channels and resale. Expected collectability can shape debt portfolio analysis, including how a portfolio is valued and segmented before collection work begins. For agencies: Score bands can help teams focus collector resources on accounts with stronger recovery potential. For first-party lenders: Managing recovery internally, the scores provide another signal to help structure decisions about which accounts to work and in what order. Look beyond collectability alone Prioritization rarely comes down to a single variable. Consider two accounts that both rank highly for expected collectability, but one has a substantially larger balance. Their recovery likelihood may be similar, but the potential dollars recovered are not. The Recovery Scores model uses debt collections analytics by combining Experian’s bureau-based score with current balance information. This helps organizations consider both the likelihood of recovery and the potential dollars at stake, while sensitive balance information remains within the organization’s working environment. Monitor accounts as they change The first decision about an account is rarely the last. Teams continually make choices about treatment, placement and resource allocation as accounts move through the recovery process. Recovery Scores is also available as a trigger. When underlying information changes, teams can receive an update and reconsider an account’s placement without waiting for the next portfolio-wide re-score. An account that shows lower recovery potential today may have a different recovery outlook later. Turn recovery potential into a portfolio strategy When portfolios contain more accounts than teams can work at once, the question isn’t simply, “How many accounts can we work?” It is, “Where can our resources have the greatest impact?” The latest Recovery Scores model helps organizations sharpen placement and treatment decisions. It gives debt buyers a way to price and segment portfolios with greater confidence and gives lenders another tool to direct limited collector time across their own post-charge-off portfolios. Learn more about Experian Recovery Scores

November 10, 2026 by Zohreen Ismail

Subscribe to our Newsletter

Enter your name and email for the latest updates.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Subscribe to our Newsletter

Don't miss out on the latest industry trends and insights!
Subscribe