When I worked as a junior analyst for one of the largest credit card issuers in the United States, the chief credit risk officer required the development of a “light switch report” and strongly encouraged everyone in her organization to read the report every day. She called it the light switch report because every morning when she walks into her office and the lights switch on, she would read the report and understand what’s going on with the business.
I took her advice and developed the habit of reading the light switch report every morning — for more than a decade while I was with the organization. I knew the volume of applications, the approval rate and the average line of credit of approvals. I developed an informed idea of how delinquency rates would look six months into the future based on the average credit score of approvals today. Her advice was valuable, and the discipline she shared helped me develop my skill sets as a junior analyst, a people manager and head of a retail business line.
Performance reports are foundational and are one of the key elements of a sound and prudent risk management framework. Regulators require effective monitoring reports and provide guidance on report generation as part of its examination process. (Office of the Comptroller of the Currency. Comptroller’s Handbook, Retail Lending Safety and Soundness. April 2017. Page 15.)
While supporting lender clients on strategy designs and development, I have an opportunity to review various performance reports. I’d like to take this time to reiterate some of the basic components of a good performance report.
Knowledge of audience is primary. Good performance reports are tailored for specific audiences who can make decisions that will affect specific outcomes. Performance reports for day-to-day monitoring would be different from reports designed for executive leadership. Transparency and accuracy are required and when reports are designed in support of areas of responsibility, those reports become meaningful and transformative.
Relevant metrics matter. Once you identify the report’s audience, the metrics you choose to appear in the report become the next important exercise. Metrics should be relevant and consistent with the audience who’s expected, upon reviewing the report, to make statements such as the business is doing well and stable, or corrective action is needed. For example, a report on the predictive power of credit risk scores intended for model developers will likely contain metrics such Kolmogorov-Smirnov (KS), Gini index or worst scoring capture rate. Such reports won’t include the average handling time of an application, which will be more appropriate for an operations team.
Metrics become even more powerful for decision-makers when calculated at a segment level. I’m a big fan of vintage reports. They tell the story of current lending practices (e.g., approval rates, average loan amount, average booked credit risk score), and more significantly they often foretell future performance (e.g., delinquency rates, charge-off rates). These foresights allow analysts and managers to plan and develop strategies today to manage the future state.
If approve or decline decisions use a dual score matrix, generate a report showing the volume of applications on the dual score matrix. It’s quicker to spot unusual distributions compared to expectations when data is presented at this sublevel. The benefit is swifter modification or new actions when needed. If statistical designs are utilized, such as test or control segments and champion or challenger segments, metrics calculated at these levels become insightful. They allow validation of a randomized process and support statistical analysis and statements.
Timeliness of reports is critical. Some reports for operational or technology purposes require constant and continuous reporting. Daily reports are important especially when new strategies are implemented. Sometimes daily reports are far more relevant within the first two or three weeks of a new strategy implementation. When daily reports show stabilization and alignment to expectations, switching to weekly or monthly reports is acceptable. Most retail products are designed for review on a cycle or monthly basis. Monthly and quarterly reports are milestones and provide good health checks of the business.
Don’t forget formats. If a picture is worth a thousand words, then use charts and graphs to display data and capture audience attention. We’re all used to seeing data presented in tables, but there are far more applications today that allow us to read reports with compelling graphics, trendlines and patterns that grab our curiosity and draw us into the story. I like narratives even if they appear as headlines on a report. Succinct comments show discipline and convey understanding of a report’s contents.
Effective performance reports evolve as the business changes. Audience, metrics and segments will change, but the basic components provide general guidelines on developing consistent and relevant reports.
