Tag: performance

When I worked as a junior analyst for one of the largest credit card issuers in the United States, the chief credit risk officer required the development of a “light switch report” and strongly encouraged everyone in her organization to read the report every day. She called it the light switch report because every morning when she walks into her office and the lights switch on, she would read the report and understand what’s going on with the business. I took her advice and developed the habit of reading the light switch report every morning — for more than a decade while I was with the organization. I knew the volume of applications, the approval rate and the average line of credit of approvals. I developed an informed idea of how delinquency rates would look six months into the future based on the average credit score of approvals today. Her advice was valuable, and the discipline she shared helped me develop my skill sets as a junior analyst, a people manager and head of a retail business line. Performance reports are foundational and are one of the key elements of a sound and prudent risk management framework. Regulators require effective monitoring reports and provide guidance on report generation as part of its examination process. (Office of the Comptroller of the Currency. Comptroller’s Handbook, Retail Lending Safety and Soundness. April 2017. Page 15.) While supporting lender clients on strategy designs and development, I have an opportunity to review various performance reports. I’d like to take this time to reiterate some of the basic components of a good performance report. Knowledge of audience is primary. Good performance reports are tailored for specific audiences who can make decisions that will affect specific outcomes. Performance reports for day-to-day monitoring would be different from reports designed for executive leadership. Transparency and accuracy are required and when reports are designed in support of areas of responsibility, those reports become meaningful and transformative. Relevant metrics matter. Once you identify the report’s audience, the metrics you choose to appear in the report become the next important exercise. Metrics should be relevant and consistent with the audience who’s expected, upon reviewing the report, to make statements such as the business is doing well and stable, or corrective action is needed. For example, a report on the predictive power of credit risk scores intended for model developers will likely contain metrics such Kolmogorov-Smirnov (KS), Gini index or worst scoring capture rate. Such reports won’t include the average handling time of an application, which will be more appropriate for an operations team. Metrics become even more powerful for decision-makers when calculated at a segment level. I’m a big fan of vintage reports. They tell the story of current lending practices (e.g., approval rates, average loan amount, average booked credit risk score), and more significantly they often foretell future performance (e.g., delinquency rates, charge-off rates). These foresights allow analysts and managers to plan and develop strategies today to manage the future state. If approve or decline decisions use a dual score matrix, generate a report showing the volume of applications on the dual score matrix. It’s quicker to spot unusual distributions compared to expectations when data is presented at this sublevel. The benefit is swifter modification or new actions when needed. If statistical designs are utilized, such as test or control segments and champion or challenger segments, metrics calculated at these levels become insightful. They allow validation of a randomized process and support statistical analysis and statements. Timeliness of reports is critical. Some reports for operational or technology purposes require constant and continuous reporting. Daily reports are important especially when new strategies are implemented. Sometimes daily reports are far more relevant within the first two or three weeks of a new strategy implementation. When daily reports show stabilization and alignment to expectations, switching to weekly or monthly reports is acceptable. Most retail products are designed for review on a cycle or monthly basis. Monthly and quarterly reports are milestones and provide good health checks of the business. Don’t forget formats. If a picture is worth a thousand words, then use charts and graphs to display data and capture audience attention. We’re all used to seeing data presented in tables, but there are far more applications today that allow us to read reports with compelling graphics, trendlines and patterns that grab our curiosity and draw us into the story. I like narratives even if they appear as headlines on a report. Succinct comments show discipline and convey understanding of a report’s contents. Effective performance reports evolve as the business changes. Audience, metrics and segments will change, but the basic components provide general guidelines on developing consistent and relevant reports.

As industry experts are still unsure when the economy will fully recover, re-entry into marketing preapproved credit offers seems like a far-off proposal. However, several of the top credit card issuers are already mailing prescreen offers, with many other lenders following suit. When the time comes for organizations to resume, or even expand this type of targeting, odds are that the marketing budget will be tighter than in the past. To make the most of the limited available marketing spend, lenders will need to be more prescriptive with their selection process to increase response rates on fewer delivered offers. Choosing the best candidates to receive these offers, from a credit risk perspective, will be critical. With delinquencies being suppressed due to CARES Act reporting guidelines, identifying consumers with the ability to repay will require additional assessment of recent credit behavior metrics, such as actual payment amounts and balance migration. Along with the presence of explicit indicators of accommodated trades (trades affected by natural disaster, trades with a balance but no scheduled payment amount) on a prospect’s credit file, their recent trends in payments and balance shifts can be integral in determining whether a prospect has been adversely impacted by today’s economic environment. Once risk criteria have been developed using a mix of bureau scores (like the VantageScore® credit score), traditional credit attributes and trended attributes measuring recent activity, additional targeting will be critical for selecting a population that’s most likely to open the relevant trade type. For credit cards and personal installment loans, response performance can be greatly improved by aligning product offers with prospects based on their propensity to revolve, pay in full each month or consolidate balances. Additionally, the process to select final prospects should integrate a propensity to open/respond assessment for the specific offering. While many lenders have custom models developed on previous internal response performance, off-the-shelf propensity to open models are also available to provide an assessment of a prospect’s likelihood to open a particular type of trade in the coming months. These models can act as a fast-start for lenders that intend to develop internal custom models, but don’t have the response performance within a particular product/geography/risk profile. They are also commonly used as a long-term solution for lenders without an internal model development team or budget for an outsourced model. Prescreen selection best practices Identify geography and traditional credit risk assessment of the prospect universe. Overlay attributes measuring accommodated trades and recent payment/balance trends to identify prospects with indications of ability to pay. Segment the prospect universe by recent credit usage to determine products that would resonate. Make final selections using propensity to open model scores to increase response rates by only making offers to consumers who are likely looking for new credit offers. While the best practices listed above don’t represent a risk-free approach in these uncertain times, they do provide a framework for identifying prospects with mitigated repayment risk and insights into the appropriate credit offer to make and when to make it. Learn about in the market models Learn about trended attributes VantageScore® is a registered trademark of VantageScore Solutions, LLC.

It seems like artificial intelligence (AI) has been scaring the general public for years – think Terminator and SkyNet. It’s been a topic that’s all the more confounding and downright worrisome to financial institutions. But for the 30% of financial institutions that have successfully deployed AI into their operations, according to Deloitte, the results have been anything but intimidating. Not only are they seeing improved performance but also a more enhanced, positive customer experience and ultimately strong financial returns. For the 70% of financial institutions who haven’t started, are just beginning their journey or are in the middle of implementing AI into their operations, the task can be daunting. AI, machine learning, deep learning, neural networks—what do they all mean? How do they apply to you and how can they be useful to your business? It’s important to demystify the technology and explain how it can present opportunities to the financial industry as a whole. While AI seems to have only crept into mainstream culture and business vernacular in the last decade, it was first coined by John McCarthy in 1956. A researcher at Dartmouth, McCarthy thought that any aspect of learning or intelligence could be taught to a machine. Broadly, AI can be defined as a machine’s ability to perform cognitive functions we associate with humans, i.e. interacting with an environment, perceiving, learning and solving problems. Machine learning vs. AI Machine learning is not the same thing as AI. Machine learning is the application of systems or algorithms to AI to complete various tasks or solve problems. Machine learning algorithms can process data inputs and new experiences to detect patterns and learn how to make the best predictions and recommendations based on that learning, without explicit programming or directives. Moreover, the algorithms can take that learning and adapt and evolve responses and recommendations based on new inputs to improve performance over time. These algorithms provide organizations with a more efficient path to leveraging advanced analytics. Descriptive, predictive, and prescriptive analytics vary in complexity, sophistication, and their resulting capability. In simplistic terms, descriptive algorithms describe what happened, predictive algorithms anticipate what will happen, and prescriptive algorithms can provide recommendations on what to do based on set goals. The last two are the focus of machine learning initiatives used today. Machine learning components - supervised, unsupervised and reinforcement learning Machine learning can be broken down further into three main categories, in order of complexity: supervised, unsupervised and reinforcement learning. As the name might suggest, supervised learning involves human interaction, where data is loaded and defined and the relationship to inputs and outputs is defined. The algorithm is trained to find the relationship of the input data to the output variable. Once it delivers accurately, training is complete, and the algorithm is then applied to new data. In financial services, supervised learning algorithms have a litany of uses, from predicting likelihood of loan repayment to detecting customer churn. With unsupervised learning, there is no human engagement or defined output variable. The algorithm takes the input data and structures it by grouping it based on similar characteristics or behaviors, without a defined output variable. Unsupervised learning models (like K-means and hierarchical clustering) can be used to better segment or group customers by common characteristics, i.e. age, annual income or card loyalty program. Reinforcement learning allows the algorithm more autonomy in the environment. The algorithm learns to perform a task, i.e. optimizing a credit portfolio strategy, by trying to maximize available rewards. It makes decisions and receives a reward if those actions bring the machine closer to achieving the total available rewards, i.e. the highest acquisition rate in a customer category. Over time, the algorithm optimizes itself by correcting actions for the best outcomes. Even more sophisticated, deep learning is a category of machine learning that involves much more complex architecture where software-based calculators (called neurons) are layered together in a network, called a neural network. This framework allows for much broader, complex data ingestion where each layer of the neural network can learn progressively more complex elements of the data. Object classification is a classic example, where the machine ‘learns’ what a duck looks like and then is able to automatically identify and group images of ducks. As you might imagine, deep learning models have proved to be much more efficient and accurate at facial and voice recognition than traditional machine learning methods. Whether your financial institution is already seeing the returns for its AI transformation or is one of the 61% of companies investing in this data initiative in 2019, having a clear picture of what is available and how it can impact your business is imperative. How do you see AI and machine learning impacting your customer acquisition, underwriting and overall customer experience?

By: Kari Michel Are you using scores to make new applicant decisions? Scoring models need to be monitored regularly to ensure a sound and successful lending program. Would you buy a car and run it for years without maintenance -- and expect it to run at peak performance? Of course not. Just like oil changes or tune-ups, there are several critical components that need to be addressed regarding your scoring models on a regular basis. Monitoring reports are essential for organizations to answer the following questions: • Are we in compliance? • How is our portfolio performing? • Are we making the most effective use of your scores? To understand how to improve your portfolio performance, you must have good monitoring reports. Typically, reports fall into one of three categories: (1) population stability, (2) decision management, (3) scorecard performance. Having the right information will allow you to monitor and validate your underwriting strategies and make any adjustments when necessary. Additionally, that information will let you know that your scorecards are still performing as expected. In my next blog, I will discuss the population stability report in more detail.