Don't Be Afraid to Fail

On a recent business trip, I read “Celebrate Failure” in American Airlines’ in-flight magazine. The president and CEO of Sygate Technologies, John De Santis , provided a number of great quotes that got me thinking about Web site optimization.

My two favorites:

I’ve been known to say that we’re not making mistakes fast enough.

There’s no way to be innovative and entrepreneurial without taking risks.

Some of the best site optimization lessons come from failed tests. Don’t be afraid of trying something in a test. Do be afraid if you aren’t using A/B or multivariate testing to improve your site. Unless you measure, test, and tune, you have no idea if changes you make improve or harm the site.

The number one problem in Web analytics is getting people to take action on the data. Assume you use analytics to identify opportunities, forecast the potential outcome of site changes, and monetize those potential changes to prioritize them. Now it’s time to act on those opportunities. Here are your options:

• Talk about improving things but don’t ever make changes.
• Roll out a new page, new content, or a new call to action.
• Conduct an A/B or multivariate test.

Obviously, the first option isn’t effective. Though the second option of just rolling out a new page, new content, or a new call to action may sound good, you have no way of knowing if it’s effective. Even if you look at how it performs the first week or two after it rolls out as compared to the previous week or two, it’s not an apples-to-apples comparison.

User behaviors frequently differ throughout the month. New campaigns, competitive pressures, and other external factors can influence behavior. This makes it difficult to know if your new content or promo is truly outperforming the original.

The biggest problem with the second option is having to guess which design or option you should use. In a recent client meeting, we presented three comps designed to improve a product detail page. The designer, copywriter, project manager, information architect, interactive head, and Web analyst were in the room. Guess who picked the design to be used. The most senior person: the head of the Web group. Instead of going with the design that pushed the envelope, he chose the one with minor changes; the safer option.

Groups that only select one option to roll out to the site are less inclined to try something extremely different or considered “unsafe.” They don’t want it to fail, kill the conversion rate, and ruin their numbers. Don’t worry about that. Take some chances. It’s the only way you’re truly going to get the performance lift you’re looking for.

The easiest way to do this is through a simple A/B or multivariate test. You can try some safe ideas and explore some riskier ones. You can use a simple A/B test on your own or a multivariate test using a tool such as Offermatica.

I’ve been involved with a number of tests in which the option that ultimately performed was very different from the original design and was considered risky. If the client had only gone with a single, safe option, it’d never have realized the performance benefits of the riskier option. If you find one of your risky attempts is failing miserably, you can easily tune the test to exclude that version. Just make sure you don’t pull the plug too soon. You can always pull it out of the test if you want. Having it there can help you expand your and your organization’s comfort zone.

There’s no way to be innovative and entrepreneurial without taking risks. Take those risks in a measurable, safe environment by testing different ideas on your live site. Learn from the wins. Learn from the failures. You’re only truly failing if you aren’t attempting to optimize site performance through live testing.