On February 21st I moderated the inaugural webinar in a series of sessions about big ideas in market research. The series is being co-produced and sponsored by Research Access and GreenBook.
The first webinar was on the topic of “Big Data.” We assembled an expert panel for this event, including:
- Steve Cohen of In4mation Insights
- Romi Mahajan of Metavana
- Lenny Murphy from GreenBook
- Charlie Wardell of Decooda
I asked the panelists whether they thought Big Data is a peril or an opportunity.
Dana Stanley: Is Big Data more a peril or more an opportunity?
Romi Majahan: Let me go ahead and take a crack at that. In my definition of Big Data, I sort of talked about both the peril and the opportunity. So let me start with the opportunity. I mean Big Data allows you to find patterns and find wisdom in a set of information. So whether it comes to figuring out how diseases spread or how to attack specific germs or viruses that sort of plague humanity, whether it’s influenza or whatever, whether it comes to trying to figure out how to really gauge the attitudes consumers have about a specific product, there’s incredible opportunity that can be found in the emerging patterns in Big Data. Similarly, the other face of that coin is that the peril is that there is so much data, that there are so many ways to interpret it, there is this issue of velocity and vastness. There’s a problem of misinterpretation or false interpretations of data. So big data allows us in some ways to both sort of capitalize on the opportunity but also sort of succumbs to the perils as well. I think it’s incredibly important, therefore, to have both a programmatic way of dealing with big data but also a somewhat scientific knowledge of what makes Big Data sort of new and different than sort of the streams of data that we used to have. So I believe there’s a peril and an opportunity, which mimics the Chinese character, of course, for opportunity, which has peril on one side and opportunity on the other side.
Steve Cohen: I’d like to take that, also. I mean I think that a good example is that you can look at, let’s say, how sales are doing over time for a particular product and you can look at those sales by geography or look at it by who the retailer is and so on. But if you’re starting to look at why does a particular skew, let’s say, the small skew versus the big skew, how do sales of that particular skew look in a rural area that is serviced by a particular retailer in a particular time of the year or the holiday season or so on. Now you’re going down and you’re finding some very micro kinds of analyses that you can do drill-downs on very easily. The problem is, if you now want to start making generalizations about what works when and where and how, you have to start modeling that data and extracting value from it. So the Big Data is to be able to look at a multidimensional array of potential predictors and factors that could cause sales to go up or to go down. As I say, you can drill down as far as you want on any of these kinds of things with a big dataset. The issue is to be able to make generalizations that you can use to generate policy for your company and directions in which you should go by doing analytics on them. So the – where we come from is that you’ve got to not only have the data but you also have the ability to summarize and make generalizations by using analytics on them.
Charlie Wardell: I mean we haven’t even scratched the surface of the scratch yet with the opportunity of data, the insights that can be gleaned out of data. I think people want more data, not less. So the opportunity is slated for $1.5 billion in 2012 for Big Data opportunities as far as the tool vendors. So this is the year where there there’s gold in those hills and everybody’s rushing to it. That’s the Big Data paradigm right now. The peril, or the things to look out for, is, ironically, information overload, analysis paralysis by data. We really need some real good insights, real good analytics. And the challenge is that how do we do that in a fast and efficient way, right? If you have the ability to analyze data very quickly, you have – and recursively and you’re able to analyze data over and over again within the same period of time, you can get really insightful analytics. That’s typically what needs to happen when you’re working with large volumes of data. The real-time nature is something that I’m really strong on, but the one thing that could be the peril is that you have a whole bunch of people out there, they’re drinking the Kool-Aid of Big Data in an architectural and methodology approach, which is a batch-based, monolithic, top-down architecture where you’re dealing with large volumes of data and, in order to get scale and redundancy, you have literally thousands of servers. And then they’re gonna find out, well, what do you do if I need real-time access to that data or if I want to act on streaming data. The whole thing about data is not only the insights and the analytics but it’s the end result, which is action. What do I do? And the people who are able to make that action quicker than others are going to seize on the opportunity.
Romi Mahajan: I want to just pop in one thing because, quoting Alexander Pope about a little learning being a dangerous thing, when I think about Big Data and when I look on the peril side, without knowledge of how to interpret it, without knowledge of where data comes from and how to analyze it and what conclusions to draw from it, we’re creating more surface area to get it wrong as well. You can take data, you can make some inferences from it, but you can also get it wrong because there’s so much data. So I really appeal to people to not simply bandy about the term “Big Data” but to really seek deep knowledge about data before they make plans based on what they believe is a correct interpretation of Big Data.
Lenny Murphy: I would add onto that, but I think the peril really is to get so caught up in the idea that we have these advanced algorithms with so much data that it answers all the questions. I don’t believe that we’re going to see that happen. The context is always going to be important, as well as the business issue. So from that standpoint, even though we can create the most advanced algorithms in the world to be able to make these relational connections between data sets and we can see the patterns, et cetera, et cetera, it’s still gonna require good, sound business acumen and training from an inside standpoint as well, I think, to be able to connect the dots and deliver real value from it. I think if we get to the point where we’re just making decisions willy-nilly because an algorithm is telling us that that is the pattern that that could be certainly the greatest path towards the peril aspect here.
Romi Mahajan: Agree. Totally agree.
Note: A special “thank you” goes out to Focus Forward for transcribing the webinar.
