Oct
06
2011

Dare to be different? Test first!

So despite the findings included in my last blog post, (indicating that subject lines with uncommon discounts do not, on average, do as well as campaigns with traditional percentage discounts), you want to test them out for yourself? I don’t blame you! As marketers we all know that what doesn’t catch the eye of one brand’s audience may engage our specific subscribers in a very different way. Plus, what fun would our job be if we didn’t try to push the envelope, even just a little bit?

To get started in testing you’ll need to:

  1. Establish the test objectiveIs it to test subject line performance, sales, or a combination of factors? Or, is it something else all together?
  1. Establish Key Performance Indicators (KPIs) such as open rate, click rate, transaction rate.
    For example, if you are only testing subject lines, then you will want to use the open rate as your success metric. For a creative test, try using the click rate
  1. Select the test method
    A/B testing or multivariate testing? It depends on if you’re changing one element vs many. Or, if you are only testing on one campaign and you want to evaluate the best subject line creative for a single campaign you can use a same day split method. This allows you to test your different versions against a small percent of your population and then send the winning subject line or creative to the rest of the population after a few hours or a day.
  1. Test to an audience
    Ensure that test cells are statistically significant based on the metric chosen to evaluate the test.
  1. Analyze your response data and determine the winner
    If the test won, then determine why. If not, then think about what you could have done differently and try again
  1. Repeat

And remember: never make assumptions or long tern business decisions on short term tests.  Make testing part of your culture and your email program will reap its benefits.


Comment are closed.