Endless Resources? Jump In. If Not, Step Back

Last week we touched on the great paradox of online data analysis: You can’t accurately analyze data until you have some sense of what behaviors you are looking for; at the same time, you have to see how folks behave before you can understand what is most important to measure.

Why, some of you have rightly asked, the conflicting advice? Is this another chicken-and-egg situation? Well, yes. It wouldn’t be — in a perfect world where every bit of data could be thoroughly examined, every pattern explored, theories drawn and then tested about every single thing that happens when a customer interacts with your Web site. In a world of unlimited computing power and unlimited time, it would not be necessary to form hypotheses before determining which data to track.

But Web sites are data hogs. Server logs of even light-traffic sites quickly compile millions of individual data points. It’s easy to follow the behavior of one Web visitor in real time (if you are set up to watch over his or her shoulder). But with hundreds or thousands of simultaneous visitors calling up any of the thousands of pages on a site, many of which can have hundreds of links to other sites, the possible pattern variations very quickly reach numbers too large for any of us mere humans to fathom. Too many zeros to count.

The best and the worst of the Web, for marketers, is the issue of scale. With good technology, your Web site can scale to do more business than any number of storefronts and can communicate with more customers than the world’s largest sales force. But then keeping track of what is happening at each step of all of those customer interactions becomes nearly impossible — and absolutely impossible if you haven’t first narrowed down the number of things to be tracked to just those actions and interactions that are likely to matter to your business.

That is why, though some readers have questioned my purpose, I’ve devoted so much attention to the design and construction of data-analysis efforts. If you have endless computing resources and scores of analysts to go through log files, ignore all this setup and just jump in. If, as is the case with most businesses, there are limitations on resources and you need to get answers quickly, a lot of preparation is necessary if good information is to come from your data-analysis efforts anytime soon.

In the course of researching this topic, by the way, I’ve had lots of conversations with the various suppliers of data-analysis solutions, and they all tell me the same thing: Clients who take the time to understand which behaviors they need to watch — those who have made the effort to think through their analysis needs before wildly collecting data — are always those who get the most useful results.

That’s not to say that the suppliers of Web-data services can’t help design the process, but they will not know your business or your customers as well as you do. They can get good results to you faster if you start the conversation after having already considered what they’ll need to know to track the behaviors that matter.