Data freshness is the key

In my last blog post I talked about how Experian’s commercial data model works.  If you haven’t seen it yet, you should give it a look at: http://www.experian.com/blogs/business-credit/2011/05/03/the-inner-workings-of-our-commercial-data/

Today, I want to explore the importance of data freshness.  Unless you are “in the business”, maintaining a commercial database might look relatively easy and straightforward.  However, those of us in the data business know that it is relatively easy to collect data, but the real work is how to keep it fresh and accurate.

We maintain a file of about 25 million active business credit files. So how does Experian manage this much data and keep it fresh and ready for our customers to decision from?

The answer is that we really push our technology hard.  Experian has invested heavily in our data environment over the last several years, resulting in a very powerful data processing engine.  Now, we are able take data in from thousands of sources and run rules against each source to make the best data elements bubble to the top. 

Let me say this again another way.  We are processing MANY sources every day, each of those sources will have every data element scrutinized relative to pre-defined, source-specific rules.  By leveraging the strengths of thousands of data sources day in and day out, our data environment has a lot to work with.

But this is only part of the data freshness plan.  We have built in many “triggers” that will set in motion automated and/or manual processes.  For example, the research team at Experian’s North American headquarters in Costa Mesa, Calif., receives the results of all news events that involve mergers & acquisitions (M&A) or other linkage-related events.  Hundreds of triggers are received each day and reviewed for relevance by the Experian team.  Over the last 12 months, 99.2 percent of the M&A updates made by the team were updated in file the same day they were announced.  All of these updates are available to customers in real-time.

So, what is the secret to keeping the data fresh?  There really is no secret – just a lot of hard work and smart folks that continue to drive our capabilities and processes as far as possible.  When you combine these powerful tools and our very talented team, we are able to make the data perform in ways that are unparalleled in the industry.  In fact, if you are a “data geek” like me it is actually rather fun!