Machine learning and Extreme Gradient Boosting

Updated: July 28, 2026 by Guest Contributor 3 min read October 24, 2018

This is an exciting time to work in big data analytics. Here at Experian, we have more than 2 petabytes of data in the United States alone. In the past few years, because of high data volume, more computing power and the availability of open-source code algorithms, my colleagues and I have watched excitedly as more and more companies are getting into machine learning. We’ve observed the growth of competition sites like Kaggle, open-source code sharing sites like GitHub and various machine learning (ML) data repositories.

We’ve noticed that on Kaggle, two algorithms win over and over at supervised learning competitions:

  • If the data is well-structured, teams that use Gradient Boosting Machines (GBM) seem to win.
  • For unstructured data, teams that use neural networks win pretty often.

Modeling is both an art and a science. Those winning teams tend to be good at what the machine learning people call feature generation and what we credit scoring people called attribute generation. We have nearly 1,000 expert data scientists in more than 12 countries, many of whom are experts in traditional consumer risk models — techniques such as linear regression, logistic regression, survival analysis, CART (classification and regression trees) and CHAID analysis. So naturally I’ve thought about how GBM could apply in our world.

Credit scoring is not quite like a machine learning contest. We have to be sure our decisions are fair and explainable and that any scoring algorithm will generalize to new customer populations and stay stable over time. Increasingly, clients are sending us their data to see what we could do with newer machine learning techniques. We combine their data with our bureau data and even third-party data, we use our world-class attributes and develop custom attributes, and we see what comes out. It’s fun — like getting paid to enter a Kaggle competition! For one financial institution, GBM armed with our patented attributes found a nearly 5 percent lift in KS when compared with traditional statistics.

At Experian, we use Extreme Gradient Boosting (XGBoost) implementation of GBM that, out of the box, has regularization features we use to prevent overfitting. But it’s missing some features that we and our clients count on in risk scoring. Our Experian DataLabs team worked with our Decision Analytics team to figure out how to make it work in the real world. We found answers for a couple of important issues:

  • Monotonicity — Risk managers count on the ability to impose what we call monotonicity. In application scoring, applications with better attribute values should score as lower risk than applications with worse values. For example, if consumer Adrienne has fewer delinquent accounts on her credit report than consumer Bill, all other things being equal, Adrienne’s machine learning score should indicate lower risk than Bill’s score.
  • Explainability — We were able to adapt a fairly standard “Adverse Action” methodology from logistic regression to work with GBM.

There has been enough enthusiasm around our results that we’ve just turned it into a standard benchmarking service. We help clients appreciate the potential for these new machine learning algorithms by evaluating them on their own data. Over time, the acceptance and use of machine learning techniques will become commonplace among model developers as well as internal validation groups and regulators.

Whether you’re a data scientist looking for a cool place to work or a risk manager who wants help evaluating the latest techniques, check out our weekly data science video chats and podcasts.

Related Posts

New data indicate signs of increased VantageScore 4.0 adoption. Recent Experian Capital Markets data reporting points to gradual growth in VantageScore 4.0 adoption, and more loans beginning to appear in a larger share of GSE pools.

November 20, 2026 by Angad Paintal
What Is a Ransomware Attack? How to Prepare and Respond

Ransomware attacks can create challenges that extend far beyond an organization’s IT systems. When critical applications become unavailable or sensitive information is compromised, businesses may face operational disruption, financial losses and difficult decisions about how to support affected customers. While cybersecurity defenses are essential, organizations also need clear response procedures, coordinated communication and recovery plans. Understanding how ransomware attacks happen, their potential impact and the steps organizations can take to prepare can help businesses respond more effectively when an incident occurs. What is a ransomware attack? A ransomware attack is a type of cyberattack in which criminals use malicious software or extortion tactics to demand payment from an organization. Traditionally, ransomware encrypts files or systems, preventing authorized users from accessing information until a ransom is paid. Some attackers also steal sensitive data and threaten to release it unless their demands are met. These tactics may occur separately or together, creating challenges for organizations trying to restore operations and protect sensitive information. Paying a ransom does not guarantee that attackers will restore access, delete stolen information or refrain from making additional demands. How do ransomware attacks happen? Ransomware attacks often begin when cybercriminals gain unauthorized access to an organization’s systems or accounts. Once inside, attackers may attempt to expand their access, steal information or deploy malicious software. Common attack methods include: Phishing and social engineering: Deceptive emails or messages that trick employees into opening malicious attachments, visiting harmful websites or sharing login credentials Compromised credentials: Stolen passwords or authentication information used to access business systems and accounts Unpatched vulnerabilities: Security weaknesses in software, servers or network devices that attackers exploit Third-party access: Compromised vendors or connected systems that provide a potential entry point Malicious downloads: Harmful software installed through deceptive websites, compromised applications or unsafe downloads Understanding these methods can help organizations identify potential weaknesses and strengthen their security practices. How do ransomware attacks affect businesses and customers? The consequences of a ransomware attack depend on the systems affected, the information involved and the organization's ability to contain the incident. Business disruption and financial losses When critical systems become unavailable, organizations may experience interruptions to customer service, payment processing, internal operations or other essential functions. The financial impact can be significant. According to the FBI's 2025 Internet Crime Report, reported ransomware-related losses reached approximately $32.3 million, up from $12.5 million in 2024. These figures do not capture the full financial impact of ransomware incidents, including business interruption, recovery expenses and other indirect costs. Customer data exposure and fraud risks Some ransomware incidents involve the theft of sensitive information, potentially exposing customers to identity theft or fraud. If attackers obtain login credentials or personal information, organizations may also need to consider related threats, including account takeover. Customer trust and reputation Customers expect organizations to protect their information and communicate clearly when incidents affect them. Delayed or confusing communication may create additional uncertainty. A coordinated response can help organizations provide timely information and appropriate support to affected individuals. What should organizations do after a ransomware attack? When a ransomware incident is suspected, organizations should activate their incident response procedures and work with qualified professionals to assess and contain the threat. Important response steps include: Activate the response team: Bring together IT, cybersecurity, legal, privacy, communications and leadership teams as appropriate. Contain the incident: Identify affected systems and take steps to limit further unauthorized activity while preserving relevant evidence. Investigate the impact: Determine which systems and information may have been compromised, including whether sensitive data was accessed or removed. Evaluate notification requirements: Work with legal and privacy advisers to determine applicable regulatory, contractual and customer communication obligations. Support affected customers: Prepare clear communications and appropriate resources when personal information has been exposed. An established data breach response strategy can help organizations coordinate customer notification and support activities following an incident. How can organizations prepare for and recover from ransomware attacks? Organizations can prepare for ransomware attacks by combining security controls, tested backups, employee training and a documented incident response plan. Recovery planning should also address how teams will restore systems, assess data exposure and communicate with affected customers. Key preparedness measures include: Maintain secure backups: Keep protected backups and regularly test restoration procedures. Strengthen access controls: Implement multifactor authentication and limit access to sensitive systems. Update software: Address known vulnerabilities through consistent patch management. Train employees: Help staff recognize phishing attempts and suspicious activity. Monitor security threats: Use appropriate detection and monitoring tools to identify potential attacks. Test response plans: Conduct exercises that evaluate technical recovery, communication and decision-making responsibilities. Following an incident, recovery should focus on restoring systems safely, addressing remaining vulnerabilities and evaluating potential risks to affected individuals. Organizations may also need to consider whether compromised information could contribute to additional fraud. Depending on the circumstances, fraud prevention solutions may support broader identity risk management efforts. Security teams should also evaluate related threats, including automated bot attacks, when assessing potential misuse of compromised accounts or credentials. Once operations are restored, reviewing lessons learned can help organizations strengthen their security controls, response procedures and future preparedness. How can breach response planning help protect customers? Ransomware preparedness should account for both operational recovery and the potential impact on customers. If sensitive information is compromised, organizations may need to coordinate notifications, respond to customer questions and provide appropriate support. Experian® offers data breach resolution services that help organizations plan for and manage customer-facing aspects of a breach, including notification management and response support. By establishing response procedures before an incident, organizations can improve coordination and prepare to address customer needs when a breach occurs. Prepare for a more coordinated breach response Explore Experian's data breach response solutions and learn how to plan for customer notification and support. Explore data breach response solutions Download the 2027 Data Breach Response Guide

November 20, 2026 by Julie Lee
How GenAI Fraud Bots Fly Under the Radar (and How to Ground Them Before It’s Too Late)

Today’s AI-powered fraud bots are exceptionally good at mimicking human behavior. Learn why a modern defense strategy is key.

November 18, 2026 by Laura Burrows

Subscribe to our Newsletter

Enter your name and email for the latest updates.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Subscribe to our Newsletter

Don't miss out on the latest industry trends and insights!
Subscribe