Big Data: Opportunity or Threat for Market Research?
Editor’s Note: A few weeks ago GreenBook and Research Access began our monthly series of webinars focused on exploring the “big ideas” in the market research industry. Our first production was focused on Big Data, and we brought together a stellar lineup of subject matter experts involved with various elements of the big data idea: business issue, role of researchers, infrastructure technology, and analytics. It was a great success. Dana Stanley of Research Access has posted a series of excerpts from that webinar here. Today Diane Liebenson, Publisher of GreenBook gives a great summary of the whole conversation. If you missed the webinar, this is a great way to catch up.
Our next webinar is scheduled for Thursday, March 29, 2012 1:00 PM – 2:00 PM EDT. The topic will be “SoLoMo: How Social Media, Localization, & Mobile are Redefining Marketing Insights. I’ll be moderating this one. This webinar will be a panel discussion of the effect of three paradigm-shifting factors have changed the way marketing insights are conceptualized, collected and analyzed: social media, localization, and mobile technology.
Our panelists are:
- Charlie Rader, Digital Insights Tools Leader, Procter & Gamble
- Steve Rappaport, ARF Knowledge Solutions Director and Author, Listen First!
- Andrew Jeavons, President, Survey Analytics
Register here: https://www3.gotomeeting.com/register/349483030
By Diane Liebenson
I generally don’t know a bit from a byte or a gigabyte from a terabyte, but when I started thinking in terms of a ‘fire hose of data’, big data started to make sense to me. Because of our growing use of the web, we now live in a world where there is a staggering amount of information available about us – including information on our preferences, our buying habits, and our whereabouts.
GreenBook and Research Access teamed up on February 16 to bring you a webinar that describes big data, its power, and its opportunity for the MR industry.
Moderated by Dana Stanley of Research Access, four expert panelists included
- Steve Cohen of In4mation Insights
- Charlie Wardell of Decooda
- Romi Mahajan of Metavana
- Lenny Murphy of GreenBook
Defining Big Data
“Big Data” is the term used to describe how advancing trends in technology will change how information is delivered to businesses. A lot more data now exists: in fact, industry pundits note that 90% of the data that exists today was created in the last two years. And with the sheer volume of social media and mobile data streaming in daily, businesses expect to use big data to aggregate all of this data, extract information from it, and identify value to clients and consumers.
One feature of big data is its ability to manage data that was too large to handle only a few years ago. The “3 Vs” illustrate why we are truly in a big data revolution:
The Volume of information. The sheer amount of data is massive.
The Variety of data. More than just structured data, big data includes unstructured data like text from social media, audio, video, GPS locator data, etc.
The Velocity of data. The speed at which data can be accessed and analyzed is exploding.
The Opportunity and the Challenge of Big Data
The promise of big data is to tell a story that nobody could tell before, and the future is about telling that story faster than ever.
There’s incredible opportunity to be found in the emerging patterns in big data. In healthcare, insight into the spread of influenza in a city can be revealed by viewing Internet search terms for the flu. Big data also enables retailers to use real-time transaction data to manage their inventory, restocking popular merchandise and marking down poor sellers more effectively than ever. That same data can be used to predict what the demand will be for new products for the next holiday season.
The opportunity of big data is also its challenge. We are drowning in data, and tweets, posts and video are not the structured data that fits well into relational databases for traditional querying. As a result, big data simply requires a new way of thinking about how to store and analyze data to accommodate these new realities and turn insights into actionable decisions.
Understanding the conversation of social media requires a new, more holistic view to make sense of the data. Text Analytics uses natural language processing techniques to “listen” to massive numbers of social media posts real-time. By applying hundreds of algorithms, metrics can be created to derive sentiment and, in some cases, the underlying emotional state of the author. Understanding emotion is the key to predicting behaviors such as whether a product will be returned to the store, whether a restaurant will get a second visit, or whether a movie will be recommended to a friend.
Big data can also be applied to video, photos and even voice. For example, emotion recognition software can photograph someone’s face repeatedly during an interview, offering marketers faster and more accurate insight on their product. How does Big Data do it? By analyzing facial muscles from hundreds or even thousands of frames and then classifying them into emotion categories.
The biggest opportunity for unstructured data is to tie it into business performance and business context. Connecting what people are saying and what a business is actually experiencing in terms of sales or other metrics would bring context to the data. Getting this right may require rethinking how the data is analyzed. Michael Wolfe at BBDO and a small number of others have been early pioneers making important contributions.
Marketers are still in the early stages of using big data. As the recent Brite-NYAMA Marketing Measurement in Transition Study revealed, many marketers have lots of customer data but haven’t given much thought yet to how they can use that data.
The Data Scientist of the Future
Perhaps the most emphatic point made by panelists was that there’s a human capital gap looming, with perhaps 100,000 data scientists needed.
It’s the rare person who possesses the combination of skills needed to tame big data. Key skills include: statistical acumen, the ability to “hack” a data set (i.e. pull it into a form ready for modeling), and a talent for turning insights into actionable business decisions.
With the need to connect structured and unstructured data and the likelihood that a very different analytical framework will be needed – perhaps a more humanistic approach that incorporates psychology and sociology – the challenge is significant. Interestingly, researchers with qualitative skills – who are good at connecting the dots and that understand business context – will become increasingly vital to fulfilling the promise of big data.
Filling the data scientist gap will require significant retooling from educational institutions, from HR departments, and from businesses. It may also require creative solutions, like employing teams of people to handle the task.
Who owns the data? Data ownership is a tricky issue, and it’s hard to talk about data ownership without also talking about privacy. Signing up for location-based services, free email, and FaceBook effectively uncovers our secrets, even long after we delete our accounts.
Different views were expressed regarding who should own the data. Companies like Coke seem more interested in hearing what people are saying about them, making the ownership issue perhaps a non-issue. But other companies may have a different plan, such as when a retailer can learn enough about you to send you coupons for baby clothes despite the fact that your pregnancy was not something you wanted anyone to know.
The Future of Traditional MR and Big Data
As to the future of MR and whether traditional tools will survive in a Big Data world, panelists suggested that big data is not the death knell of traditional MR, but traditional MR may shrink in size and be subsumed within the Big Data tent.
There will always be a need to use traditional MR and researchers will need to match the right tool to the problem they are trying to solve. Additionally, unless Big Data is specifically better – like in a fast-moving situation where it makes sense to tap into social media data to extract information around a brand or to judge who is going to win the Super Bowl– traditional MR is likely to be a straightforward and easy choice.
Several references were recommended for those who want additional reading to understand big data, its risks and opportunities:
The Big Data Glossary by Pete Warden
The Information Diet by Clay Johnson