Test It Out

Okay, I’ll admit it: I’m a testing junkie. Email tests, that is. I just can’t get enough of ’em those helpful little cells of knowledge in which you split this, segment that, add this, take out that…

Sure, they may not always be the easiest things to set up, AND they can make reading results a lot more complicated. But, boy, can you learn a lot from them. Not to mention the fact that they can help your subsequent and future email campaigns truly soar.

Testing can be as simple as breaking up a list into three parts and emailing each group a unique subject line. Or it can be as complex as setting up several different “panels,” each containing various lists with like-minded names. The goal is then to read and apply results based on relative strength from list to list, panel to panel.

For purposes of this two-part article, we’ll take a look at both extremes.

First, the tried and true: Let’s say you want to set up the simplest of tests. Perhaps you do indeed want to test three or more different subject lines. Or you want to pit different offers against one other. The easiest way to start is by creating a grid:

TEST CELL	VARIABLE	QUANTITY	KEYCODE	CHECKPOINT: MAILDATE, OFFER, COPY, ETC. IS IDENTICAL?
Cell A	Subject Line: Version 1	5,000	0710V1	Yes
Cell B	Subject Line: Version 2	5,000	0710V2	Yes
Cell C	Subject Line: Version 3	5,000	0710V3	Yes

Keep in mind that if you’re using a solutions provider for deployment, it will most likely plug in a unique tracking code of its own for your “call to action” link within each cell. However, if you’re sending internally, a date/version keycode such as the one above can be helpful because after the results are final, it’s easy to tell immediately which one of your cells is the winner without having to consult yet another spreadsheet. And, as far as the checkpoint goes, if you’re testing one variable at a time, it’s critical to make sure all other variables are the same. This makes for more accurate results.

For reporting purposes, depending on which measurement you’re using (i.e., click-through rate [CTR], conversion, cost-per-click or sale, or all of the above), all you need to do for a simple test such as the one above is add columns for those results. Then simply insert the actual final numbers when the promo has reached completion. In this case, we’re looking at an offer that has, as its final conversion goal, a product sale in the amount of $59.

So the end of your final grid may look something like this:

TEST CELL	CTR %	TOTAL CLICKS	CONVERSION (% OF CLICKS)	NEW SALES	TOTAL SALES $
Cell A	6.7%	335	2.3%	7.7	$454.30
Cell B	8.2%	410	1.4%	5.7	$336.30
Cell C	6.4%	320	2.2%	7.04	$415.36

Granted, this is an overly simplified example (we’ve got to start somewhere), but the results are clear. As far as dollar volume goes, Cell A is the winner. However, if this campaign’s success were to be measured by click-through rate alone, the winner would be Cell B. Meaning that the results of a catchy subject line and/or a strong CTR are not always what they appear. Another reason why testing and laying out even the easiest of strategies can be key.

When all is said and done with a test such as the one above, you’ll know where to go. At least as far as your subject line goes. But what if you want to learn about more than just one variable? What if, in fact, you want to learn not only what your best offer is, but what out of several promotions are your most compelling creative, your most responsive lists, your top subject lines, and more, all from a single campaign?

Then, my fellow email marketers, it’s time to get down and dirty…

But let’s save that for next week. The plain and simple truth: We’re outta time. So see you then.