Sample, Social Media & Emerging Techniques, Oh My!
It’s been one of those weeks when a lot has been happening, both in my little corner of the research world and in the broader industry. All of the activity is encouraging and exciting; I daresay that our industry may …
It’s been one of those weeks when a lot has been happening, both in my little corner of the research world and in the broader industry. All of the activity is encouraging and exciting; I daresay that our industry may be on the brink of growth again!
I am attending the AMA MRC this week here in my hometown of Atlanta; it will be interesting to hear from our colleagues what their sense of things is. I’ll be blogging and tweeting on the event each day. so be sure to check back here for the latest.
As I walked the exhibit hall this evening, I was struck by a few things:
- The exhibitors were sample provider heavy; all of the major players are there. Further evidence of the steep competition in that space, with little proof on who has the best approach (although Mktg Inc. has made a good start)
- The schedule seems a little light on emerging technologies and techniques. No presentations (or exhibitors) on Mobile MR, and just a bit on Social Media, Neuromarketing, etc…
That brings me to a few related items of news I’d like to bring to your attention. Maybe it’s serendipity, but the news this week that caught my attention is focused around sample and social media! The Blogging Gods must be smiling upon me!
First, researchers from the University of Vienna and the University of Deusto in Bilbao, Spain, released the results of a study concluding that researchers should keep online survey questions short and simple. That seems like kind of a no-brainer, but yet many in our industry don’t embrace this simple idea. That is unfortunate, because the evidence is stacking up that our current way of doing things is not working well. For example, The Harvard Business review just published an article on the current state, and future direction, of research titled Reading the Public Mind. It’s a fairly scathing review of the failure of our industry to adequately adjust to the new norms of consumer communication preferences. Here is an excerpt:
To Stanford political scientist Douglas Rivers, the prognosis is clear: “We’re going to see the death of telephone polling.” As an alternative, pollsters are looking toward online panels of volunteer respondents. Web surveys, which are already used by virtually all consumer researchers, cost about one-tenth as much as phone surveys, and it’s easier to query a volunteer panel than to question a random sample of the populace.
But online panels bring their own serious problems: While the general population is becoming less and less eager to respond to surveys, the most active web panelists are, strangely, more eager. Some of these “professional respondents,” as pollsters call them, volunteer for 15 to 20 surveys a month. They’re also demographically unrepresentative of the general population—more educated and more likely to be white, and more likely to be either young or old (but not middle-aged). Plus, Tourangeau says, there is some evidence that many web respondents do a sloppy job of answering the survey questions, racing through so that they can earn their meager financial rewards with as little work as possible.
The American Association for Public Opinion Research, a standards-setting body made up of nearly 2,000 survey professionals from private and government organizations, takes a dim view of web panels that are drawn from volunteers. In a report this year, the group declared that survey results from these panels aren’t “projectable to the general population.”
Though marketers use surveys to bounce ideas off potential consumers rather than to take the pulse of the masses, the nonrepresentativeness of web panels is an issue in business, too. Kim Dedeker, an executive with the Kantar market research network and a former head of Procter & Gamble’s Consumer & Market Knowledge department, acknowledges that time and again, P&G would get a go-ahead on a product concept from an online panel, only to see disappointing results when the product was field-tested. “If the data that we’re getting from our research isn’t replicable and predictive, then we may as well not do it at all,” she has written.
Stuck between traditional polling, which is getting more difficult and expensive, and cheaper surveying of unrepresentative panels, the field is due for a radical rethink. At least one thing is clear: In their quest to probe the public mind, survey firms will make greater use of the vast amounts of digital data that people wittingly or unwittingly provide about themselves.
Thankfully, some firms are listening and are working hard to develop new techniques that do address the issues of convenience samples as well as respondent tolerances for surveys. CivicScience, (whom I’ve written about before) just released a whitepaper on The CivicScience Model for Quantitative Attitudinal Research. This is nothing short of revolutionary folks, and it’s the type fo thinking our industry desperately needs right now. Take a look:
CivicScience reaches millions of individuals from a diverse set of demographics, consumer segments, and geographies. These individuals answer short-form, one to three-question polls, embedded in commonplace polling applications across a variety of third-party web properties, including news sites, blogs, social networks, membership organizations, and mobile-based locations. Each respondent’s profile accumulates longitudinally by linking their many sets of answers to a unique anonymous identifier. As a respondent’s profile grows, the delivery of new questions is optimized to build an increasingly valuable profile of attitudes, beliefs, preferences, and demographics. Aggregated and organized responses produce daily, insights into consumer sentiment across broad segments of the population.
…The global population of CivicScience respondents and their profiles are dynamic, rich, and constantly growing. In the first nine months of 2010, we cataloged over 40,000,000 poll responses with many respondents providing dozens and even hundreds of answers over time. By the end of 2010, we are on pace to collect over 3,000,000 poll responses per day, projecting toward over 2 Billion responses cataloged in 2011.
Smarter folks than me can debate the science of their approach, although based on the brains behind this company I think anyone will have a hard time poking holes into the model. Download the whitepaper for yourself and you be the judge, but for my money this is an important next step in resolving the issues related to panel representativeness and respondent engagement.
CivicScience is just one example; there are certainly many more companies (and more entering the space every day it seems) that are using innovations around mobile engagement, social media monitoring, and data mining to launch creative new ways to collect and distill consumer insights. Two firms exhibiting at the AMA MRC come to mind as good examples of new approaches to social media research: Crimson Hexagon and Intrepid. What is interesting about these two firms is their different approaches to social media research. Crimson Hexagon claims to go beyond pure sentiment analysis common for most SMR platforms:
Based on groundbreaking work conducted at Harvard University’s Institute for Quantitative Social Science, Crimson Hexagon is the only commercially available opinion analysis platform that lets you distill meaning – with mathematical precision – from the sea of opinion, information and data available in social media throughout the web.
…Our technology analyzes the entire social internet (blog posts, forum messages, Tweets, etc.) by identifying statistical patterns in the words used to express opinions on different topics. Based on this insight, it goes beyond simply measuring positive versus negative. Instead, it delivers a clear understanding of the nature of the online conversation – the key themes, their relative size, and how they change over time –with up to 97% accuracy.
Intrepid takes a different approach and combines SMR with traditional methods using a full service model. What is most interesting about them is that they were recently acquired by the quickly growing Alterian, a marketing engagement technology provider. Here is a glimpse into their strategy and value proposition:
Intrepid has been acquired by Alterian, a global provider of marketing engagement solutions. Alterian has acquired Intrepid to accelerate its existing Social Media Monitoring solution, SM2, to provide a packaged solution of social media analytics and market research capability to organisations who have neither the time or in-house expertise to understand what the social media world is saying about them and how to make best use of it in their businesses.
These two approaches also are indicative of the great debate happening in the MR industry right now on how to get a handle on social media research and the issues involved with this rapidly growing new field. On Friday Annie Petitt of Conversition and Ben Smithee of Spych Market Analytics squared off in The Great Debate, sponsored by The Global Online Moderator Community. The crux of the debate was around whether researchers should engage directly consumers using social media tools, and if so, how we should do it. Annie was of the opinion that we should not due to privacy concerns, while Ben believes that we have to engage with consumers more directly in order to maximize the value of SMR. Both made a lot of great points, and it’s well worth a listen; you can access a recording of this engaging and informative debate here.
On a related note, various trade associations are working to try to develop some standards around social media research.Here is info on the next upcoming event from the MRA:
Date: Tue, Sep 28, 2010
Time: 1:00 PM ET-2:00 PM ET
Medium: Live Webinar
Speaker: Annie Pettit, Jim Longo and Ray Poynter, Conversition Strategies, iTracks & The Research Place
PRC: 1 Contact Hour in Marketing Research
Session Description: Free Webinar. Get a sneak peak at this Buzz Session that will take place at MRA First Outlook Conference this November. This preview session will highlight the areas that IMRO volunteer members have identified as important areas and problems in the field, as well as describe IMRO’s approach to building standards and guidance around those areas. Session leaders will describe the types of social media research, IMRO’s guidelines development process and note how session attendees can ensure that their expertise and knowledge are included in the process.
I’m not a big fan of this approach; I think we have too much potential to hobble ourselves by applying guidelines that may fit into the classical view of MR but that have little grounding in the business reality of our industry right now. That said, any work to legitimize SMR as a viable research technique is a good thing, so I am happy with the effort from that perspective.
To wrap up, Tom Ewing on his Blackbeard Blog reports on the recent MRS Social Media Research 2010 conference. I think he sums up the entire state of where our industry is on a variety of issues better than I can, so here is Tom’s take:
Much like last week’s ESOMAR gig, the Social Media Research conference suggested to me that social media research is in a bit of a holding pattern at the moment. People are doing it, case studies are starting to trickle in, the broad methodological outlines are accepted, nothing terribly radical is being said (or not by researchers – see later).
That’s OK! That’s healthy in fact. If there’s nothing really exciting to say and everyone’s just getting on with things that’s not a BAD state of affairs by any means. I was talking to Ray Poynter at ESOMAR and he remarked that a lot of the big debates on Twitter and LinkedIn aren’t really getting a lot of conference airtime – it was noticeable that at SMR 2010 there was hardly any talk of ethics and industry codes, for instance. But some things live better in the virtual coffeehouses of research’s chattering classes than on a conference stage where there wouldn’t be time to do them much justice anyhow.
I suspect this period of relative conference calm won’t last too much longer anyway. The case studies will get meatier (and have a bit more ROI information in them); the mobile applications will get sexier; the backlash will grow, and we’ll all be arguing like old times this time next year. For now, there’s bits and pieces to chew on, and pretty much every presentation I saw today had something that got me interested. The single statement which got most Twitter buzz was John Griffiths “unique is hopeless” – at the moment, with the category still tiny and getting established, we need to share with each other more than we need to compete.
Let’s see if that is what comes out of the AMA MRC this week. That’s it for now, but look for regular posts over the next few days.