AB testing is an excellent tool in the whole cycle of optimizing your marketing strategies. Your newsletter mustn’t be forgotten. Make sure your newsletter is of added value to your audience and of high quality. Of course, there’s always something to improve. You can make improvements based on intuition, but why not test that intuition first by AB Testing your newsletters?
In this post, I’ll dive into newsletter AB tests by explaining what you can test. I won’t discuss testing examples, but I will tell you what aspects to pay attention to when trying.
Newsletters AB Testing
Subject line
With most email campaign tools, you’ll have the possibility of AB testing the subject line. That means you’ll be able to give your newsletter a number of different subject lines. If you try 2 different subject lines, ordinarily, 50% of the subscribers receive the first variation, and the other 50% get the other variation.
A/B testing your subject lines is only relevant for testing your open rate, not your click rate. The available rate is the percentage of how many successfully delivered newsletters were opened by your subscribers. The click rate percentage, on the other hand, gives insight into how many successfully delivered newsletters registered at least one click. The subject line won’t make a difference in your click rate since it doesn’t affect anything within the body of your email. That said, testing your subject lines is still very important, as you want as many people as possible to read what you have to say. So you want your subscribers to open your newsletter, right?
One set of rules that our friend Jordie van Rijn (a great email marketer) taught us, which has greatly helped us, is C.U.R.V.E:
- Curiosity: try to pique the readers’ interest by asking them questions.
- Urgency: create urgency by having limited-time offers or offering things that need to be done now.
- Relevance: Ensure you’re putting the most relevant content to your audience in your subject line.
- Value: Convey the newsletter’s value by offering something exclusive (this can be an exclusive product offer but also exclusive content).
- Emotion: Use punctuation, such as exclamation marks, to elicit emotional responses from your readers.
From name
Another thing you can almost always test is your name. This is precisely what it sounds like: the word that shows from whom the emails are coming:
This is, again, something that will only affect your open rate. However, this is an aspect that people tend to forget because it’s such a small thing to change. However, the form name can be pretty significant. It’s the first thing people see when your email arrives, so it had better be good. Testing this will make sure it is.
Send time
I’m unsure whether all email campaign tools offer this A/B testing option, but MailChimp does. You can test what send time (MailChimp calls this “delivery time”) works best for your audience. You must do some work here beforehand because you’ll have to decide when the variations go out yourself.
So, try to find out when most of your emails are opened or when most of your audience is awake. Especially if your emails go to an international group of people, like ours, this might be a good thing to test. Sending your emails at the right time can result in more people seeing your newsletter and getting invested.
Content Newsletters AB Testing
Content is the big one. This is where you can go all out and test anything you like. Everything within the content section of your email can be tested, and that’s a lot. You must consider what you want to try and treat these A/B tests like any other. We’ve written a post explaining this: Hypothesize first, then test. In any case, you must try only one aspect at a time. Otherwise, you can’t tell which part of your A/B test caused a higher click rate.
I always prefer to begin with this one because it’s furthest from the subscribers’ process of receiving, opening, and reading a newsletter. I test content first because I don’t want to optimize a part of my email (say, the subject), while what the readers see next (like the email’s content) could undo all the optimization I did before.
Just a few ideas of what you could think about when you want to test your email’s content:
- Your email’s header;
- An index summarizing your email;
- More (or less) images;
- The different tone of voice;
- More buttons instead of text links;
- More ideas on Jordie’s blog.
Before testing
When you start testing, most email campaign tools will offer you two options:
- Send your variations to your complete list, or
- Send your variations to a percentage of that list, declare a winner and then send the winner to the remaining people who haven’t received a newsletter.
I’d strongly urge you to use the first option. Let me tell you why. First, sending multiple variations to just a sample of your list means you’re cutting down on ‘respondents.’ You’ll have more data when you send it to the complete list. And that means your results will be more reliable.
However, if your list is big enough, this probably won’t matter much. I’d still choose the first option because using the second option. The winning variation gets sent out hours (or even days) later. Especially for newsletters, this can be problematic because, at this point, it’s not really “news” anymore. Using the second option also means you can’t determine when the email will be sent. And, as I’ve already said: send time can be pretty significant.
If timing is less critical to the emails you’re sending out, and you have an extensive list of subscribers, you could choose the second option. In that case, the remaining people on your list will always get the winner, which could be beneficial.
Results of Newsletters AB Testing
So you’ve thought up some brilliant variations of your newsletter’s content, its subject, from the name or send time. Time to send out that newsletter! Once you’ve sent it, there’s nothing more you can do. You must wait until the first results trickle (or flood) in. Make sure you take notice of the differences in outcomes. Which version got the highest open rate? Which version had the best click rate?
When comparing results, click rate always has my priority. After all, a high click rate means your readers will probably end up on your site, where you have many more opportunities for selling, for example. However, we also always use custom campaigns on all the links in our newsletter. And since we’ve set up eCommerce tracking in Google Analytics, we can see which version of our newsletter generated the most revenue. If you have a business to run, that’s probably the metric you want to see increasing.
Unless you’ve set up some kind of eCommerce tracking within your email campaign tool, this metric won’t be available in their results. So don’t value the results of these tools too much. Make sure you focus on what’s essential for your business and check those metrics.
Also: don’t be too quick to judge. I usually wait for a few days, up to a week, before I conclude because many people will still be opening and engaging with your email after a few days.
Happy testing!
What do you think of the steps and rules we’ve set for ourselves? Do you have similar ideas that you follow? Or maybe something completely different? Let us know in the comments!
