The dos and don’ts of subject line testing

It’s an all-too-common pitfall: subject line testing is often the result of last minute deliberations. It begins as a smorgasbord of subject line options, then a debate occurs over which is best, and the “test” becomes the two favored lines pitted against each other. This test will produce a winner, but if the subject lines are essentially unrelated, will it provide any insight into what strategies to build upon?

Therein lies the tricky part. The lines need to carry essentially the same message, with only a subtle difference between them. Strategy examples include: short vs. long, personalization vs. no personalization and broad language vs. a specific offer (for extra guidance and inspiration, please also refer to our white paper It’s all in the wording: A guide to optimizing your email subject lines).

Our white paper provides tons of great “dos,” but what about the missteps to avoid? We gathered some of our experts to get their insights on having a carefully devised, carefully implemented game plan when it comes to subject line testing.

Planning ahead and persistence is key

Megan Dandois, Experian Marketing Services’ Associate Director of Cross-Channel Marketing, is always praising the importance of having a long-term plan for subject line testing. She notes that clients who have taken a disciplined approach to testing – and stick with the results for an optimal period of time – have seen response rates climb and subscriber lists stabilize. “It’s tempting to abandon a plan too soon, especially when trends change in the industry. For example, when symbols in subject lines became popular last year, many companies hopped on the bandwagon, but that might have come at the expense of a subject line plan with more potential for their brands.”

Don’t be afraid to mix things up

There’s also the tendency to depend too heavily on sales and promotions, with the assumption that a number in the subject line is always the way to go. “This is an obvious choice,” says Dandois, “But what happens when there’s no sale? You want insight into the kinds of subject lines that intrigue your audience even without a promotion. That’s where strategy comes in. That’s where testing really pays off.”

No promotion? No problem. Consider creating lines with the following themes: 

  • Curiosity: Questions, riddles and unfinished thoughts evoke a sense of mystery in the subscriber so they will want to read more. 
  • Fun: Humor, puns, plays on words and provocative language are used to surprise subscribers and entice them to keep on reading. 
  • Direct: Clear and to the point, direct messages evoke urgency and succinctness. 
  • Relationships: Relatable messages help subscribers identify with the products and associate them with groups or special interests. This can be done by utilizing social media, special events, member newsletters or group buying sites.

Which category is the “best”?



Source: Experian Marketing Services’ It’s all in the wording: A guide to optimizing your email subject lines white paper

Keep deliverability top of mind

Changing strategies mid-stream also poses a threat to deliverability. ISPs can be suspicious of unrecognizable subject lines, whereas building a strategy that gradually improves open rates over time generates more activity and more engagement. This helps your communications earn familiarity and avoid the spam filter.

Katie Van Wyck, Client Partner at Experian Marketing Services, points out that deliverability concerns are especially important regarding the new Gmail message tab setup, which designates incoming emails into folders such as Primary, Promotions, Social, Updates and Forums. “Most of our clients’ messaging will end up in the Promotions folder, which may not get checked as often as the subscriber’s Primary folder. A subscriber can designate certain promotional emails to go to the primary folder, but this requires a few extra steps. However, if a subscriber regularly looks forward to hearing from your brand, he or she will make those changes. That’s why you want subject lines to hit the mark with your audience.”

Approach your testing as an art and a science

The tendency to avoid or rush through testing strategies is understandable, says Van Wyck, but there are solutions. “Some clients are inclined to change their subject lines last minute since their messaging is tied to last minute promotions. Because of the nature of their business, many don’t know how to approach testing systematically. In other instances, they will change a subject line so significantly that it doesn’t mirror the line it’s competing against. We try to stress that one minor detail can cause a boost. Too many alterations make it difficult for us to determine what factor caused better performance. This has to be approached as a science.”

Speaking of experimentation, when done correctly, data can be extracted from testing literally word for word. “Subject line testing is a very effective way to find out what key words or phrases resonate with your audience,” says Experian Marketing Services’ Client Partner, Vivian Swertinski. “Consider that your subject lines are being read by those that opened and those that chose NOT to open the email. That is good news, as there is great opportunity to move the needle and make a big impact on mailing metrics.” She adds that subject lines can help build brand image, especially if you incorporate a style that is conveyed through the subject lines, creative and product offering. “Not surprisingly, a brand’s customer base often has similar style and personality. Consider testing more than two subject lines at a time to gain insight into the style and phrasing that really connects with your customer base.”

Avoid variables that skew results

Experian Marketing Services’ Client Relationship Manager, Matt Meola, also suggests ways to steer his clients’ subject line tests to be more scientifically and statistically accurate. “Always consider the time of day you are sending your test. You can’t expect a true reading when you send it between 3 and 6 AM, when most people are not awake. It’s too small a sample audience and it may result in a false positive.”

Sending during work hours is more viable these days thanks to the boom in mobile usage (rest assured clicks and opens from mobile activity count), and he also advises clients to consider the size of their test population. “If you have a small list, avoid doing a 10/10/80 split; perhaps use a 15/15/70 to get a truer reaction. And, leave yourself enough time to gather enough data. This is a common oversight – tests are rushed and their analysis, even more so. There are so many potential insights to be gleaned and shared, but you have to invest time and strategy into it.”

Remember – it can be simple if you want it to be

A great and unique subject line that resonates with your audience can result in a big surge, but without strategy, the momentum gained off that line will be lost the next time. Before you roll out a campaign, put the right game plan into effect. Achieve success with properly executed subject line testing.


About Marketing Forward

Marketing Forward is more than a name, it’s a quest. Through data-driven insights, news articles and thought pieces, our experts share their perspectives on the latest marketing and consumer trends, keeping you informed and inspired to think differently and push your marketing strategy forward.

Latest Tweets