Analyzing A/B Testing Email Marketing Data
Welcome to the wild world of A/B testing emails! If you are responsible for email marketing, you know that every subject line, word choice, and link placement are variables that affect your email performance. Not to mention, factors such as send time, day of week, month, etc.
The crazy thing is, the performance is all measured, right down to the last click.
If you’re like most marketers, you’re probably spending dozens of hours testing different variations of the same email to:
Identify factors that are most effective at driving conversions and engagement.
Discover the type of content that resonates with your target audience.
Learn which version is highly likely to avoid the spam filter.
But there’s just one problem with A/B testing your emails. You generate a lot of data, especially if you have a large email list.
Your marketing automation tool isn't likely the best tool for data exploration and analysis. Therefore, you are going to want to export the data, open the CSV online, and use a friendly spreadsheet format for analysis.
What is the best way to Analyze it? Well, let’s have a look.
A/B Testing Email Marketing Examples
I pitched this idea to my close friend Will Andrews, who’s in charge of Product and Marketing at Gigasheet. He collaborated with me to build a dummy A/B email testing dataset – where we together came up with:
1,000 random emails
We added an email variation column group - where we’ve set the entries between 1 and 5 for five different email variations. We randomly assigned email variation numbers to 1,000 emails in our list.
We randomly assigned “Yes” or “No” entries to the following column groups:
Opened (Yes or No) – To understand which version of the email helped us achieve the highest open rate.
Clicked-Through (Yes or No) – To understand which version of the email helped us achieve the highest click-through rate.
Responded (Yes or No) – To understand which version of the email our recipients replied to the most.
Booked a Meeting (Yes or No) - To understand which version of the email helped book the most meetings.
Unsubscribed (Yes or No) – To discover which version of the email helped us get the lowest unsubscription rate.
Here’s a sneak peek at our dataset:
Now, let’s analyze it!
Using Google Sheets to Analyze My A/B Test Email Campaign Data
I work as a freelance content partner for Gigasheet. But that doesn’t mean I hate Google Sheets or Microsoft Excel.
In fact, I love them!
To conduct my analysis, I loaded my spreadsheet to Google Sheets, which you may find here.
If you’re like me, you already know how complex Google Sheets is. From giving you the ability to play with spreadsheets and perform calculations to advanced features like conditional formatting, data validation, data filtering, and sorting, data visualization, and more, Google Sheets is incredibly complex.
Every time I had to apply a filter or formula – I had to look it up via “Help.”
While you’ll soon get the hang of the platform (know all about its features), it takes time. Google Sheets has a steep learning curve.
First, I wanted to sort my data by email variations. I’ll admit – it was simple – all I had to do was create a filter and sort the data in the email variation column by “Sort A -> Z.” You can also hide or unhide column groups in Google Sheets easily.
But then came the hard part.
I wanted to find out – which version of my email had the highest open and click-through rate. I thought of applying a filter – but if I went that route, I’d have to count open and click-through rates for every email variation – which is just too much work.
Well – you can always apply grouping – but when I tried doing it in Google Sheets – I found out that it was one of the hardest things on the planet. Using Google Sheets to group your data is like getting a toddler to eat their veggies – it’s HARD.
On top of this, did you know that – Microsoft Excel and Google Sheets have their own limitations? At present, Google Sheets can only process 10 million cells. Excel’s row and column limits stand at 1,048,576 and 16,384 respectively.
Also, if you try to access a large spreadsheet file using Microsoft Excel or CSV, it may result in your PC crashing or browser/software freezing. I have been known to cause Excel to crash!
That’s the reason we encourage people to use Gigasheet.