Fanning the Flames of Big Data

Last week, an article in TechCrunch offered an opinion on “Why We Need to Kill Big Data.” Perhaps it was inspired by this tweet that was embedded in the article:

“I hate the term ‘big data’. Its all just data. Someday we’ll all just call it data – but for now it’s big.” -Arun Jacob of Disney at #BDBN

— Dr. Martin Wells (@BigDataInsights) September 13, 2012

Perhaps big data as a buzzword is used too often and inappropriately, but please don’t kill the idea. I don’t know of a term that became such a crossover catchphrase based on anything analytics-oriented. The term is no longer reserved for quants. It generates a lot of buzz and captures non-analytics people’s imagination. If this catchall phrase can get more people interested in analytics, then we should support it.

The author argues that “enterprise companies like IBM, large retailers, financial services giants and many others have been parsing through massive amounts of data for some time now, before this word was even coined.”

While that is true, there are a few things about what is properly called big data that make it not-the-same-old-same-old:

1. Until recently, massive amounts of data were analyzed by sampling. Companies looked at subsets of data and made assumptions based on the results of their analysis. With today’s data toolset, companies can analyze this massive volume of data all at once.
2. Most often that data was analyzed after the fact. Today, the data can be analyzed in real time. This allows agile organizations to act on the data to generate additional business value out of it, no matter the velocity at which it comes into the business.
3. All the data that was being crunched was structured data. The kind of stuff that you could input into an Excel spreadsheet. Today, companies like RetailNext are making it easy for retailers to analyze unstructured data like in-store video. Companies are also analyzing gobs of unstructured data like phone conversations, images, etc. We have a great new variety of unstructured data to analyze.
4. Most importantly, big data can now become the great equalizer for smaller businesses as well as larger ones. The cost structure for using these data tools has now made it more widely available and accessible to a greater number of businesses.

As quoted in The New York Times:

Rod A. Smith, an I.B.M. technical fellow and VP for emerging Internet technologies, says he likes the term because it nudges people’s thinking up from the machinery of data-handling or precise measures of the volume of data.

“Big Data is really about new uses and new insights, not so much the data itself,” Mr. Smith says.

At the end of the day, if we can help fan the flames of big data, maybe CEOs would start to trust CMOs and their teams, CMOs would become data analytics-driven, claim their unfair advantage, and we would see the CMO job tenure last longer.

Big data, I’m a fan!

Nerd image on home page via Shutterstock.