To Win with Machine Learning, It Isn’t What You Do; It’s How You Do It

by Jim Bander 6 min read April 24, 2019

If you’re a credit risk manager or a data scientist responsible for modeling consumer credit risk at a lender, a fintech, a telecommunications company or even a utility company you’re certainly exploring how machine learning (ML) will make you even more successful with predictive analytics. You know your competition is looking beyond the algorithms that have long been used to predict consumer payment behavior: algorithms with names like regression, decision trees and cluster analysis. Perhaps you’re experimenting with or even building a few models with artificial intelligence (AI) algorithms that may be less familiar to your business: neural networks, support vector machines, gradient boosting machines or random forests. One recent survey found that 25 percent of financial services companies are ahead of the industry; they’re already implementing or scaling up adoption of advanced analytics and ML.

My alma mater, the Virginia Cavaliers, recently won the 2019 NCAA national championship in nail-biting overtime. With the utmost respect to Coach Tony Bennett, this victory got me thinking more about John Wooden, perhaps the greatest college coach ever. In his book Coach Wooden and Me, Kareem Abdul-Jabbar recalled starting at UCLA in 1965 with what was probably the greatest freshman team in the history of basketball. What was their new coach’s secret as he transformed UCLA into the best college basketball program in the country? I can only imagine their surprise at the first practice when the coach told them, “Today we are going to learn how to put on our sneakers and socks correctly. … Wrinkles cause blisters. Blisters force players to sit on the sideline. And players sitting on the sideline lose games.”

What’s that got to do with machine learning? Simply put, the financial services companies ready to move beyond the exploration stage with AI are those that have mastered the tasks that come before and after modeling with the new algorithms. Any ML library — whether it’s TensorFlow, PyTorch, extreme gradient boosting or your company’s in-house library — simply enables a computer to spot patterns in training data that can be generalized for new customers. To win in the ML game, the team and the process are more important than the algorithm. If you’ve assembled the wrong stakeholders, if your project is poorly defined or if you’ve got the wrong training data, you may as well be sitting on the sideline.

Consider these important best practices before modeling:

  • Careful project planning is a prerequisite — Assemble all the key project stakeholders, and insist they reach a consensus on specific and measurable project objectives. When during the project life cycle will the model be used? A wealth of new data sources are available. Which data sources and attributes are appropriate candidates for use in the modeling project? Does the final model need to be explainable, or is a black box good enough? If the model will be used to make real-time decisions, what data will be available at runtime? Good ML consultants (like those at Experian) use their experience to help their clients carefully define the model development parameters.
  • Data collection and data preparation are incredibly important — Explore the data to determine not only how important and appropriate each candidate attribute is for your project, but also how you’ll handle missing or corrupt data during training and implementation. Carefully select the training and validation data samples and the performance definition. Any biases in the training data will be reflected in the patterns the algorithm learns and therefore in your future business decisions. When ML is used to build a credit scoring model for loan originations, a common source of bias is the difference between the application population and the population of booked accounts. ML experts from outside the credit risk industry may need to work with specialists to appreciate the variety of reject inference techniques available.
  • Segmentation analysis — In most cases, more than one ML model needs to be built, because different segments of your population perform differently. The segmentation needs to be done in a way that makes sense — both statistically and from a business perspective. Intriguingly, some credit modeling experts have had success using an AI library to inform segmentation and then a more tried-and-true method, such as regression, to develop the actual models.

During modeling:

  • With a good plan and well-designed data sets, the modeling project has a very good chance of succeeding. But no automated tool can make the tough decisions that can make or break whether the model is suitable for use in your business — such as trade-offs between the ML model’s accuracy and its simplicity and transparency. Engaged leadership is important.

After modeling:

  • Model validation — Your project team should be sure the analysts and consultants appreciate and mitigate the risk of over fitting the model parameters to the training data set. Validate that any ML model is stable. Test it with samples from a different group of customers — preferably a different time period from which the training sample was taken.
  • Documentation — AI models can have important impacts on people’s lives. In our industry, they determine whether someone gets a loan, a credit line increase or an unpleasant loss mitigation experience. Good model governance practice insists that a lender won’t make decisions based on an unexplained black box. In a globally transparent model, good documentation thoroughly explains the data sources and attributes and how the model considers those inputs. With a locally transparent model, you can further explain how a decision is reached for any specific individual — for example, by providing FCRA-compliant adverse action reasons.
  • Model implementation — Plan ahead. How will your ML model be put into production? Will it be recoded into a new computer language, or can it be imported into one of your systems using a format such as the Predictive Model Markup Language (PMML)? How will you test that it works as designed?
  • Post-implementation — Just as with an old-fashioned regression model, it’s important to monitor both the usage and the performance of the ML model. Your governance team should check periodically that the model is being used as it was intended. Audit the model periodically to know whether changing internal and external factors — which might range from a change in data definition to a new customer population to a shift in the economic environment — might impact the model’s strength and predictive power.

Coach Wooden used to say, “It isn’t what you do. It’s how you do it.” Just like his players, the most successful ML practitioners understand that a process based on best practices is as important as the “game” itself.

Related Posts

New data indicate signs of increased VantageScore 4.0 adoption. Recent Experian Capital Markets data reporting points to gradual growth in VantageScore 4.0 adoption, and more loans beginning to appear in a larger share of GSE pools.

November 20, 2026 by Angad Paintal
What Is a Ransomware Attack? How to Prepare and Respond

Ransomware attacks can create challenges that extend far beyond an organization’s IT systems. When critical applications become unavailable or sensitive information is compromised, businesses may face operational disruption, financial losses and difficult decisions about how to support affected customers. While cybersecurity defenses are essential, organizations also need clear response procedures, coordinated communication and recovery plans. Understanding how ransomware attacks happen, their potential impact and the steps organizations can take to prepare can help businesses respond more effectively when an incident occurs. What is a ransomware attack? A ransomware attack is a type of cyberattack in which criminals use malicious software or extortion tactics to demand payment from an organization. Traditionally, ransomware encrypts files or systems, preventing authorized users from accessing information until a ransom is paid. Some attackers also steal sensitive data and threaten to release it unless their demands are met. These tactics may occur separately or together, creating challenges for organizations trying to restore operations and protect sensitive information. Paying a ransom does not guarantee that attackers will restore access, delete stolen information or refrain from making additional demands. How do ransomware attacks happen? Ransomware attacks often begin when cybercriminals gain unauthorized access to an organization’s systems or accounts. Once inside, attackers may attempt to expand their access, steal information or deploy malicious software. Common attack methods include: Phishing and social engineering: Deceptive emails or messages that trick employees into opening malicious attachments, visiting harmful websites or sharing login credentials Compromised credentials: Stolen passwords or authentication information used to access business systems and accounts Unpatched vulnerabilities: Security weaknesses in software, servers or network devices that attackers exploit Third-party access: Compromised vendors or connected systems that provide a potential entry point Malicious downloads: Harmful software installed through deceptive websites, compromised applications or unsafe downloads Understanding these methods can help organizations identify potential weaknesses and strengthen their security practices. How do ransomware attacks affect businesses and customers? The consequences of a ransomware attack depend on the systems affected, the information involved and the organization's ability to contain the incident. Business disruption and financial losses When critical systems become unavailable, organizations may experience interruptions to customer service, payment processing, internal operations or other essential functions. The financial impact can be significant. According to the FBI's 2025 Internet Crime Report, reported ransomware-related losses reached approximately $32.3 million, up from $12.5 million in 2024. These figures do not capture the full financial impact of ransomware incidents, including business interruption, recovery expenses and other indirect costs. Customer data exposure and fraud risks Some ransomware incidents involve the theft of sensitive information, potentially exposing customers to identity theft or fraud. If attackers obtain login credentials or personal information, organizations may also need to consider related threats, including account takeover. Customer trust and reputation Customers expect organizations to protect their information and communicate clearly when incidents affect them. Delayed or confusing communication may create additional uncertainty. A coordinated response can help organizations provide timely information and appropriate support to affected individuals. What should organizations do after a ransomware attack? When a ransomware incident is suspected, organizations should activate their incident response procedures and work with qualified professionals to assess and contain the threat. Important response steps include: Activate the response team: Bring together IT, cybersecurity, legal, privacy, communications and leadership teams as appropriate. Contain the incident: Identify affected systems and take steps to limit further unauthorized activity while preserving relevant evidence. Investigate the impact: Determine which systems and information may have been compromised, including whether sensitive data was accessed or removed. Evaluate notification requirements: Work with legal and privacy advisers to determine applicable regulatory, contractual and customer communication obligations. Support affected customers: Prepare clear communications and appropriate resources when personal information has been exposed. An established data breach response strategy can help organizations coordinate customer notification and support activities following an incident. How can organizations prepare for and recover from ransomware attacks? Organizations can prepare for ransomware attacks by combining security controls, tested backups, employee training and a documented incident response plan. Recovery planning should also address how teams will restore systems, assess data exposure and communicate with affected customers. Key preparedness measures include: Maintain secure backups: Keep protected backups and regularly test restoration procedures. Strengthen access controls: Implement multifactor authentication and limit access to sensitive systems. Update software: Address known vulnerabilities through consistent patch management. Train employees: Help staff recognize phishing attempts and suspicious activity. Monitor security threats: Use appropriate detection and monitoring tools to identify potential attacks. Test response plans: Conduct exercises that evaluate technical recovery, communication and decision-making responsibilities. Following an incident, recovery should focus on restoring systems safely, addressing remaining vulnerabilities and evaluating potential risks to affected individuals. Organizations may also need to consider whether compromised information could contribute to additional fraud. Depending on the circumstances, fraud prevention solutions may support broader identity risk management efforts. Security teams should also evaluate related threats, including automated bot attacks, when assessing potential misuse of compromised accounts or credentials. Once operations are restored, reviewing lessons learned can help organizations strengthen their security controls, response procedures and future preparedness. How can breach response planning help protect customers? Ransomware preparedness should account for both operational recovery and the potential impact on customers. If sensitive information is compromised, organizations may need to coordinate notifications, respond to customer questions and provide appropriate support. Experian® offers data breach resolution services that help organizations plan for and manage customer-facing aspects of a breach, including notification management and response support. By establishing response procedures before an incident, organizations can improve coordination and prepare to address customer needs when a breach occurs. Prepare for a more coordinated breach response Explore Experian's data breach response solutions and learn how to plan for customer notification and support. Explore data breach response solutions Download the 2027 Data Breach Response Guide

November 20, 2026 by Julie Lee
How GenAI Fraud Bots Fly Under the Radar (and How to Ground Them Before It’s Too Late)

Today’s AI-powered fraud bots are exceptionally good at mimicking human behavior. Learn why a modern defense strategy is key.

November 18, 2026 by Laura Burrows

Subscribe to our Newsletter

Enter your name and email for the latest updates.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Subscribe to our Newsletter

Don't miss out on the latest industry trends and insights!
Subscribe