The best emails are the ones that get the most results for your business. Here’s the kicker—even your best emails can get better. All it takes is a bit of experimentation and iteration.
Enter A/B Testing. It pits one email against another. The results help identify the winning elements in each. You can find better subject lines, boost open rates, and close more sales.
But how do we know which element to use in the A/B test? What email metrics should we focus on? And how can we transform analytical data into actionable information?
We’ll answer all of that and more! First, we need to go over the basics:
- What is A/B testing?
- What email elements should we A/B test?
- Tools required to conduct A/B tests
- A/B testing examples
- How long should an A/B test last?
- A/B test best practices
What is A/B Testing for Email Marketing
A/B testing determines which emails are “objectively” better for your business. You can A/B test 2-3 variations of your control email to find which elements work best for a specific lead segment.
Let’s say you want to send a cold email to 20,000 B2B leads. Instead of sending the same email sequence to all the leads, you can split it into two groups.
Emails sent to Group A could have a formal and concise subject line, while the ones for Group B could use emojis. After a week, Group B had 10% more opens.
So, in the next round of emails, you send the same subject line to Group A and get even better open rates. That’s what A/B testing is about—finding what works best.
It doesn’t even stop at subject lines. There are email elements we should test depending on how far leads are in our sales pipelines.
Email Elements We Should A/B Test
Don’t go overboard with A/B testing. Try and stick to one or two elements at a time. Too many variables lead to inclusive results. Use this cheat sheet to know exactly what to test:
A/B Testing for Top-of-Funnel Leads
We want to test email elements that improve our open rates. Think of subject lines, preview text, from addresses, or sender names.
For cold emails, sales subject lines should be engaging, personalized, and relevant. Variations of these subject lines could include a different tone, adding a GIF, or using first names.
A/B Testing for Middle and Bottom-of-Funnel Leads
Middle and bottom-of-funnel leads are familiar with your brand and actively engage with your emails. That means testing out elements that improve our click rates.
These include elements within your email copy, such as overall brand voice, email icebreakers, or how you write your call-to-action.
If you’re trying to convert newsletter subscribers, you can try different images, using human photos, stock vs. genuine, or including sales video letters.
Tools Needed To Conduct A/B Testing
Most email marketing software has integrated A/B testing features. But most also lock you into A/B testing entire campaigns.
Instantly, instead of just your regular A/B testing, you can do A-Z testing. Users get to test what subject line, CTA, or value proposition works best for the same email sequence and targeting.
But you can still compare entire campaigns and view metrics in real-time in Instantly’s Analytics dashboard. With the right tools ready, it’s time to start A/B testing your emails.
How To A/B Test Using Instantly
For this example, we’ll use Instantly’s A/B testing features. Instantly, users can test several variations of email sequences, each with unique subject lines and email copy.
But as mentioned, don’t go overboard. Try out 2-3 and see what works. Add more as you start to scale or when you want to fine-tune campaigns.
First, create a new campaign. Then, go to the “Sequences.” On the left side, you can add new variants for each step of your email sequence, including subject lines and your email copy.
Next, you can add variants for your follow-ups or next steps. If you want the emails threaded, keep the “previous email’s subject.”
You can test as many variances as you want in a campaign. With the “Auto Optimize” option, Instantly’s algorithm determines your best-performing variant based on your “winning metric.”
This can automatically switch off poor-performing variants. But if you want to do it manually, you can do so within the analytics tabs or from the sequence editor.
To A/B test entire sequences, you’d have to create two or more campaigns and see which ones perform better in the analytics tab.
Top-Down A/B Testing Examples You Should Try This 2024
At the end of the day, you can test as many variables as you want. But if you wish your A/B tests to be systemized, you can try a top-down approach. That means testing variables based on where your prospects are in the sales funnel.
Subject Lines for Cold Emails
We want prospects to open our first email, especially during cold outreach. Subject lines are the best way to do so. The better the subject lines, the higher the open rates. Here are some variations you can try:
- Adding personalization on the subject line (first names or company names)
- Using emojis or no emojis
- Shorter vs. longer subject lines (consider preview text)
- Capitalization (This is a subject line vs. This Is A Subject Line)
- Negative vs. Positive (How to quickly lose subscribers vs. How to gain subscribers fast)
- Questions vs. Answers (Can {{pain point}} be solved? Vs. The solution to {{pain point}})
- FOMO vs. no FOMO (3 days left on 60% discounts Vs. Save 60% on {{product}})
From Lines or Sender Name
Authority and trust play a significant role in email marketing. Would you want to open an email from Company Name or Dave from Company? So, try these variations for your from lines:
- Company name: Your Company
- Company Department: Marketing
- Team member's first name only vs. full name
- Team member's first name with the job title vs. full name
- First name and company: Dave from Your Company
Take note that the sender name used should depend on your lead segment. For example, you could use “Company blog” as the sender name for newsletter subscribers.
Meanwhile, prospects in your cold email outreach campaigns might respond better to sender names that appear more personal instead of corporate.
Email Preview/Email Snippet
When you’re crafting emails, consider what the preview looks like. From the preview alone, prospects can already see if an email is another generic cold outreach.
The majority of email users likely use their mobile devices to view emails. Don’t forget to see if it cuts off. Ensure that prospects see something engaging and valuable.
You can try out variations such as:
- Copy from the first line of the email.
- A summary of your unique selling point
- Your call-to-action (CTA)
The preview is usually the first line of your email body. You can customize the preview in your email using a bit of HTML coding. But be careful when using custom HTML. It can cause issues.
Plain Text vs. HTML
There’s always a debate about Plain Text vs. HTML in email marketing. While HTML allows for a ton of customization, Plain Text is the way to go for deliverability.
For cold emails, plain text is recommended. It gives a more personal feel. For newsletters, HTML can work, primarily if you’re focusing on design/branding.
However, as mentioned, using HTML might cause missing display elements or broken links. So, be sure to test your emails first before sending them to your subscribers.
Images vs. No Images
Images, GIFs, or sales letter videos can make your email more engaging. But that’s not always the case. This depends on your prospect demographic, industry, and the image’s relevancy.
You can even A/B test the image used. Maybe memes from a specific show work better? Still, copy comes first.
For newsletters, images, and graphic design play a key role in conversions. Here, you can deliberate on:
- Using stock images vs genuine photos
- Adding a human element to the images
- How the images are laid out
Email Body
How you structure or format your email copy affects how prospects perceive it. This affects sentence structure, what you include in your introductions, or even the use of bullet points.
A lot of fine-tuning can be done when you A/B test your email body. Here are some variations to consider:
- Using bullet points
- Longer vs. shorter email copy
- Using icebreakers or quick company intro
- Straight to the point vs. casual
Remember that your primary email copy should hook prospects in the first half, pique interest in the middle with your unique selling points, and lead them toward an actionable CTA.
Call to Action (CTA)
Are we still testing CTA button colors? Probably not. Instead of little psychology tricks, we can focus on how we construct our CTAs and overall design.
Note that CTAs are drastically different for cold email and newsletter prospects. For cold emails, you can try:
- Asking for a meeting vs. asking for a reply
- Adding a calendar link vs. using an exact date
- Using low-effort CTAs (No-brainers like free case studies or infographics)
- Using links to a sales video letter expanding on your USPs.
Newsletters can be more flexible and flashy with CTAs. Some of the variations to consider are:
- Shorter vs. longer CTAs.
- Capitalization “Grab Your Discount Today!” vs. “Grab your discount today!”
- CTA font size or button size
- Contextualizing CTAs (Start Automating Email Marketing Today vs. Start Your Free Trial)
Delivery Day and Time
When’s the perfect time to send an email? During a lunch break? At the break of midnight? 9:42 AM on a Tuesday? The answer depends on your prospects.
Every website that has statistics on this says something different. Your prospects could very well be the same. Since we can’t pinpoint an exact time, we can play around ranges.
For example, Between 9 AM and 12 Noon or 10:30 PM - 12 Midnight. Consider the day you’re sending the email as well. You can even test sending on holidays vs. not sending on holidays.
How Long Should an A/B Test Last?
As a rule of thumb, you’d want your A/B tests to run for the duration of your sequence. But with tools like Instantly, you can A/B test elements within campaigns to see which ones work best.
Studies show that for metrics like open rates, 2 hours is enough to see which emails are better. You can even wait it out for 12 hours and still have a statistical significance of 90%.
Clicks go well with open rates. But sometimes, prospects take time to consider your offer. So, around 3+ hours is the sweet spot, producing a 90% statistical significance.
However, we must still account for prospect behavior variances—things we can’t control. For example, they could take a day to open your email.
Prospects might not even read your first email but will open your follow-up. To get a better sample size, running your campaign for two weeks (or how long your sequences are) might give you more accurate data.
Key Takeaways
A/B testing emails give you an objective view of which emails perform best. It helps you identify winning subjects from lines, copy, and CTAs, making it the best way to iterate and improve your best-performing emails. To make the most out of your A/B testing, consider these best practices:
- A/B test 2-3 elements or variations at a time for a more focused result.
- Set benchmarks and goals to identify which email metrics are most important.
- A/B test subject lines, preview texts, or sender names to improve open rates.
- Test email copy, brand voice, and CTAs to improve click rates.
- Leverage email marketing software that can automate A/B testing for you.
Ready to find your best email? Instantly can help you test email variations from A to Z! Try it out today!
