Email Marketing: Campaign Analysis, Metrics, Best Practices

With all the sexiness oozing out of social media it might seem insane to write about email. It’s been relegated to the “OMG that cesspool of spam that no one cares about because everyone is using Google Wave and Facebook!”

Not true. Email remains an immensely credible and profitable channel, with an immense reach to boot. To not have it as an active part of your marketing portfolio is sub-optimal.

The only requirement is that your mental model (and indeed, company culture) should be solidly rooted in permission marketing. Every fiber of your being. Every thought that crosses your mind. Every person in your company. Embrace permission marketing and email will be a surprising and loyal BFF. I dare say even more than Search (and without a shred of doubt, more than Social Media). Because you control everything. The message, the customer data, the ability to reach current and prospective customers, drive new sales as well as repeat sales, experiment with new ideas and offers, and so much more.

So… Email = totally worth dating, engaging, marrying and having babies with.

You just have to have the right mental model (see Seth Godin above) and you have to… wait for it… wait for it… measure everything you do! Just to ensure you are executing against your right mental model.

Analyzing email campaigns requires three important insights:

1. You must use metrics that are unique to the medium.

2. You can’t track everything. Sorry.

3. You need to think end-to-end, and not just your silo.

Seems simple enough, right? Amazingly, and sadly, I see examples of failure simply because the above three “simple” things are not clearly understood or executed against.

One of the core challenges with email is that you have to deal with multiple data sources. There are three primary sources:

1. Your campaign data. How many emails went, to whom, what happened to them, yada, yada, yada.

2. Your website data. What happened after someone clicked on your email links?

3. Your company cross-channel outcomes data. Multi-channel customer purchase behavior, customer lifetime value. You know, Analysis Ninja territory!

This means that the key enabler of gorgeous analysis is your ability to ensure that for #1 your email campaigns are tagged correctly (with tracking parameters), for #2 your website landing pages are tagged completely (with the correct JavaScript tag) and finally, for #3 you’ve thought of the primary key that you need to pass into your backend database.

Do that, and you’ve met the minimum requirements and the possibility of achieving awesomeness.

Ready for the best email marketing campaign metrics?

Macro Perspective: End to End means ABO.

For everything you do it is important to measure your effectiveness of all three phases of your effort:

Acquisition. Behavior. Outcomes.

ABO. I’ve found it increasingly easy to use the ABO acronym to ensure that I’m thinking end to end.

If your measurement effort is missing one of those three, you’ll be less successful than you deserve to be. So for our email campaign analysis let’s look at metrics using that framework.

Optimal Acquisition Email Metrics.

You’re trying to measure how well you are doing to: Send emails. Get people to read them. Click.

The key metrics you’ll measure will typically come from your email service provider (so make sure you check they can provide these metrics before you sign up and fork over the cash!).

A secondary thing would be to ensure, or your mom will really be mad, that *every* link in your email is tagged with campaign tracking parameters your web analytics solution needs (without this you can’t do the B and O analysis). Most good email providers will do this automatically for whatever web analytics tool you use. Here’s a post detailing vendors and process for Google Analytics: Tips for Tracking Email Marketing Campaigns.

Beyond the standard tactical stuff like number of emails sent etc., here are the metrics I’d recommend to analyze your acquisition greatness:

Delivery rate = (# of emails sent – # of bounce backs) / # of emails sent

This is your bread-and-butter outcome metric when it comes to your campaign; it answers the following simple question: did we stand a chance at success? Note that the increasing use of junk and spam boxes means that bounce backs are not the cleanest way to measure deliverability. The emails might have just ended up in the junk email box where they never stood a chance of being opened. But life and lemons and lemonade. Right?

[Bonus, from Yahoo!: What can I do to ensure that my email goes to the right folder?]

[Update: John Duffy, in comments below, has a great tip: Segment delivery rates by Email Service Provider (ESP). Try different ones every now and then and see which one works best.]

Up next, the excitement of the first engagement:

Open rate = # of emails opened / # of emails delivered

Allow me to rush and point out that this metric is usually just directionally accurate. Most email programs now have preview panes that typically block images and scripts (Outlook, Thunderbird, Gmail, everyone), and default settings prevent data collection due to concerns about viruses. That is okay. Over time compare rotten apples to rotten apples. Even if only directional, this metric is of value, for example in helping you understand how effective the single most impactful thing in your email campaign is: the subject line (no not the offer!).

And now to the reason we actually sent the emails:

Click-to-deliver rate (CTDR) = # of clicks / # of emails delivered

IMHO this is a key measure of the quality of your email list, and of the effectiveness and relevance of your message. Segmenting this metric is really powerful. You can learn whether text messages or messages with images get a higher CTDR. You can compare customers in California, Idaho, and Florida; new and existing customers; or various demographics, etc., etc., etc. This should drive aggressive experimentation of email content / offers / targeting / every facet by your team.

Finally let’s not forget a very, very important signal of our email marketing effectiveness:

Subscriber retention rate = # subscribers – bounce backs – unsubscribes / # subscribers

How much do you stink? This is perhaps as strategic an analysis as you could do for your email campaigns. Here you are measuring both the technical effectiveness of your email campaigns over time (reducing bounce backs) and the relevance of your messages and the targeting of the same (reducing unsubscribes). Measure retention rate over time in aggregate – or for optimal health, segment retention rate – and measure it for the various objectives you have set for your email marketing program.

Four simple metrics that help you understand how effective you are in the acquisition stage of your email marketing.

But there is more…

Optimal (Website) Behavior Email Metrics.

Analysts will almost always measure emails sent, and will often measure some type conversion signal. But it is rare that we obsess about what happens between the email click and the possible conversion. Did the website deliver the goods? What worked in terms of website content and customer behavior? In your analysis, don’t skip this part because what happens after that email click will either deliver high conversions or kill the most valuable offer you’ve ever sent.

The challenge is to figure out what behavior to track.

Here would be an obvious choice:

Bounce Rate = # of email campaign visits with a single Page View / # of email campaign visits

Simple reason: “Never let your campaigns write checks that your website cannot cash.” That’s really what you are measuring. Are your landing pages delivering on the promise you made in the email campaign? It does not matter if you have a 100 percent response rate on your email campaign if the website Bounce Rate for the campaign is 99 percent. This metric helps you find opportunities for immediate improvement – such as pages and calls to action you should test, and content that fails to deliver.

You’re still not safe even if the bounce rate is 0%, but at least now you are in the game to get valuable business outcomes.

For non-ecommerce content based websites here’s another obvious behavior metric:

Depth of Visit = percent of email campaign visits that last longer than xx pages

The xx in the definition forces you to think up front and plan for site behavior before you send an email blast and, of course, measure your performance against that goal. The assumption here is that more page views is more ad views or more future visits or deeper brand impact or… think that through up front as well.

For ecommerce websites here’s a behavior metric:

Actions Completed = percent of email campaign visits that watched a video

Watched a video or searched for a store location or added to cart or reached the Technical Specs page or customized a car or rated our products or logged in using Facebook or Google+ to get product recommendations based their friends circle or played a treasure hunt… so many other things. I’m using the term action generically. You can use Event Tracking or Custom Variables to capture customer behavior that creates value for your business (online or offline).

Depending on your type of website, other metrics will also be important to you. Bottom-line: do not ignore web behavior; it is an important part of measuring your email campaign success.

Optimal Outcomes Email Metrics.

I’ve declared, frequently, that I have OOD (Outcomes Obsession Disorder). If you are not reporting deeply on the outcomes from *every* digital effort…. why even exist?

With our email campaigns, that’s even more true. If you spend money on acquiring traffic you’d better be delivering 50x (or whatever low bar you want to set for yourself 🙂 return on investment.

You can start with the most overused metric in web analytics:

(Macro) Conversion Rate = # of One Big Thing / # of email campaign visits

One Big Thing could be ecommerce sales orders, the number of leads received, downloads of your product, sign up for trials, people volunteering to protest, donations to your church, new accounts opened for your new social network… anything. In my earlier posts you might have seen me refer to this as “your Macro Conversion,” the most important thing to your business when you use email marketing.

Oh, and don’t forget to segment it like crazy. Be ruthless at identifying causes for low performance.

It might seem logical that you’ll also measure the second most overused web metric:

Average Revenue per Email Sent = total revenue / # of emails sent

I am very fond of this metric because it stresses productivity. Notice that you can play some really nice games by substituting the denominator with # of emails read or # of emails delivered. But my perspective is, why settle for lower standards? Isn’t the point of our email campaign to get maximum value? So, let’s set a high bar. Using # of emails sent will force quality in your email campaigns because it will mandate that the list be very clean and targeted. It is important in your journey as an analyst or marketer that you influence the positive behavior of your company by choosing your metric definitions wisely.

[Here’s another example of using a metric’s definition to change behavior: Excellent Analytics Tip #16: Brand Evangelists Index]

But why measure only 2% of the outcomes on your website? Why not measure the complete business value delivered by your email campaign visitors? Measure the macro AND micro conversions:

Micro Conversion Rate = # of Many Small Things / # of email campaign visits

Many Small Things are other things your campaign traffic can accomplish on your website. Write product reviews, follow you on Twitter / Facebook / Flavor of the Month, download your Android app, sign up for an email newsletter (see the circular reference? :)), sign a petition, sign up for an affiliate account, and so many things that add value to your business. More ideas here and here.

Now you’ve measured 100% of the business value delivered by your email marketing efforts.

And speaking of that… if I could only measure only one metric for my email campaigns, this is it:

Average Economic Value per Email Sent = total economic value / # of emails sent

Economic value is our obsession. Not revenue. It encompasses macro plus micro conversions. It will ensure your job promotion and a happy marriage. I won’t go into too much detail about how to do that. If you are interested in a promotion (and a happy marriage!) just hop over here: Excellent Analytics Tips #19: Identify Website Goal [Economic] Values.

Finally for those of you measuring revenue, and think of yourselves as Analysis Ninjas, you are measuring this delightful metric (don’t let me down!):

Profitability = (revenue generated – campaign cost – cost of goods sold) / # of emails sent

Most email Marketers and Analysts will measure revenue and order size and other such obvious metrics. But we rarely spend time measuring profitability. In my experience that is normally because it is hard to find data related to true costs. In the case of email, that means the cost of the campaign (the cost of buying the list, sending the email, using resources, and so forth), as well as the cost of creating the products and services (cost of goods sold).

The difficulty in getting the numbers (bug Finance!) should not stop you from trying to measure Profitability along with Revenue. It is very easy to imagine that the most successful email campaign in the history of your company could very well cause bankruptcy (Costs greater than Revenue), and campaigns that look like mediocre performers could be most profitable. See why this is important?

Email marketing works. You just need to resist the temptation to abuse your customers. Don’t pre-select “sign up for our email” boxes and have an extra step to confirm opt-ins; always think of the customer benefit and not just what you will gain. You should have a big, clear, one-click unsubscribe link on top of every email. But most of all, you need to be relevant. That’s really all it takes: treating your customers exactly as you would like to be treated.

Oh, and computing the above metrics to optimize your email marketing. That’s it.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s