Testing your online properties? Focus on the user

Data-driven marketing is the Holy Grail for success in marketing today. No one is happier than a marketer who has been given the OK to test, fail and iterate based on real-time data. Add in the velocity and volume of big data and you can’t fail! Or can you?

As more consumers move online to digital channels, marketers are relying more on A/B testing to collect user interaction data and make decisions. Based on how individuals react to a given change online, marketers can make adjustments that will yield the best conversion results.

The problem we are seeing is that companies are acting too quickly to action their A/B test results, without regard to given business goals and objectives. When marketers move on results too quickly or without a plan, this introduces scope creep, or uncontrolled changes that are outside of the original project plan.

While marketers need to be agile, they need to make sure they have a coordinated strategy that does not sacrifice the long term product and company vision. By making changes too quickly, they may adjust an online property without appropriate review.

Secondarily, too many tests introduce a fragmented experience for users. Often, companies will conduct A/B tests based on individual hits rather than considering each user as unique. This means someone may hit an online property one time and have one user experience, come back and see a totally different site.

Techcrunch recently pointed out Twitter’s “new” user interface which differed session to session to measure individual’s user interactions. You can note in the photo below the introduction of sharing icons underneath the posts on the left side.



As a marketer, A/B tests have value in their aggregated data results; but as the photo shows, you can’t forget the data represents individual’s online experience. By fragmenting the individual, you jeopardize user experience – a fatal mistake within a web-driven business.

Now, as a web marketer, I’m not saying you shouldn’t A/B test — they are very valuable and yield measurable results that can make a real difference. But, as you A/B test, you need to ensure two things:

  • You maintain the integrity of every user’s experience
  • The variables you’re testing (and proposing to change) align with your product and overall company strategy

As we know, “with great data comes great responsibility.” Just because you can test something, doesn’t mean you should … at least not without aligning the outcome to your business strategy.


About Marketing Forward

Marketing Forward is more than a name, it’s a quest. Through data-driven insights, news articles and thought pieces, our experts share their perspectives on the latest marketing and consumer trends, keeping you informed and inspired to think differently and push your marketing strategy forward.

2015 Digital Marketer

2015 Digital Marketer Report

Everything you need to know about digital marketing: 1,000 global marketers give best practice examples and insights on the latest trends.

Download the report now >>

Latest Tweets

Follow us