1. Clear Seas Research
  2. SIS International Research
  3. RN_ESOMAR_WP_GBook_480_60
  4. TopBanner480x60_Greenbook

Mobile Qualitative – How Does It Fit In The Research Toolkit?

Can technology experts, however user-experience focused, create platforms that truly help the touchy-feeling world of Qualitative Research? If Revelation is an example, then it appears so.



Editor’s Note: Edward Appleton is doing a series of posts focused on the client-side view of mobile research, with an emphasis on use cases and best practices learned so far. This is the fifth  post in that series that we’ll be publishing over the rest of the month. Parts 1 – 4 can be found here.


Edward Appleton

If you’d asked qualitative researchers – in Europe at least – what they thought about online qualitative methods or platform possibilities ten, or even five years ago, you may well have drawn a blank stare, a pause, followed by reasons why face to face is actually superior.

Add “mobile qualitative” to that, and the reactions are likely to have been similar, perhaps even more intense.

Revelation (http://bit.ly/Lv4efy), a US Oregon-based research software company, has arguably lead the way showing skeptical, skilled but partially tech-averse qualitative research practitioners how technology – notably Smartphones – can be used to help enrich and enhance a multi-modal qualitative research design.

Founded in 2007, Revelation currently employs 25 people, is in growth mode, and is expanding its international reach – their mobile app is currently available in 16 languages.

As an innovator in the mobile qual. space, and an Agency that counts Procter and Gamble as one its clients, it’s the sort of New Market Research Company – driven by technology, scale and a visionary approach – that is currently revolutionizing and improving Market Research.

Start-up mode is definitely behind Revelation, as is the validation/proof of concept phase. Their research platform is increasingly being used by Researchers the world over. Its white label offering allows easy-to-do customized branding; 80% of the Company’s work is providing a platform for Researchers. This makes the question of whether Revelation is effectively the Intel of the mobile qual world, “powering” qualitative research experiences, so to speak – a tantalizing one.

It also raises the question: can technology experts, however user-experience focused, create platforms that truly help the touchy-feeling world of Qualitative Research?

I first met Steve August, Revelation’s CEO, at an ESOMAR Conference in Valencia, then again in Ghent, Belgium. I took the opportunity to chat with him about Revelation’s approach to and experience with online qualitative and more particularly mobile market research.

Why Mobile?


I blame my wife for all this” is Steve’s tongue-in-cheek response to my question as to how Revelation originally got started on its path to mobile qual.

The origins of Revelation are an interesting example of how successful innovation is born – taking two approaches or worlds – digital technology and established qualitative Research practice and protocol, then merging the two to create something synergistic.

Steve’s spouse Kimberly – the founder of Revelation’s legacy company, KDA – was an ex-Fitch Design user experience research professional who had chosen to go freelance as from the mid-1990s. Specializing in immersive in-depth qualitative rather than Groups, she often worked in Diary studies as a facet of in context research, executing traditionally using pencil-and-paper formats. These were copied, posted out, the content sifted through manually at analysis stage. A mixture of the mechanical and the meticulous, time consuming and with elements of non-added value cost.

Steve’s route to mobile research was more indirect and partly fortuitous. He describes himself as a creative technologist – having worked variously as a producer of multi-media CD Roms, documentaries, and as a Business Intelligence Consultant. He occasionally happened upon his wife’s Research diaries at the copying stage, and questioned if existing technology could make the process more efficient.

Online blogging and web-diaries were at the time – 2002 – just becoming part of the web landscape, with increasing numbers of internet users choosing to engage and collaborate online.

Why not take MR diaries online?

The potential to transform pencil-and-paper data collection seemed to make sense – the benefits stood out:

  • Eliminate non-added value logistical activities (copying, postage)
  • Participants could upload photos, videos when equipped with flip-cams
  • Real-time reporting
  • Easy sharing
  • Quicker analysis through text tagging.

The combination of these sounded enticing, game-changing even, but: would respondents share their feelings online? Would the technology work?

An early stage pilot aimed to address these questions.

Piloting Qualitative Mobile: “Understanding Parenthood Better”


Steve and Kimberly had recently become parents, and chose a topic close to their own heart – if or how becoming a parent changes personal identity, and if the sense of freedom is diminished or lost.

The design adopted was projective – participants were equipped with a flipcam, blogging software and asked to find and upload images of themselves both before and after parenthood, then comment on what had changed.

Paired-friend recruitment was chosen as a method – with the aim of accessing both individual and shared responses.

Mobile: a simpler Triangulation Option

The pilot worked well – the technology performed without glitches. Respondents shared a wealth of emotional responses, indicating that they were not inhibited by the medium.

The study also revealed how the online mobile medium allows researchers to pursue different insight avenues more easily through digital sequencing.

Dads were interviewed about their experience with exactly the same questions, then Mums, and then both were “confronted” with the others’ viewpoint – leading to surprises as well as confirmation.

This sort of sequencing is relatively easy to execute in online mobile, offering an efficient route to first-stage triangulation.

Advantages of Mobile Qualitative over Traditional Qualitative Techniques.

mobile anywhere

The Parenthood pilot study indicated that digital technology could indeed play a new, complementary role in qual. research. This lead to further validation studies, each of which highlighted the difference to face-to-face qualitative – Focus Groups, In-depth-Interviews.

Mobile qual. “took us into people’s living rooms” to use Steve’s words, allowing participants to comment in their own time. Software also offered the option of making comments personal or open to a group – allowing peer-to-peer interaction, and the beginnings of a community-style set-up.

Revelation’s core argument for online and mobile qual. is built on the simple insight – that most of the interesting things happen in participants’ lives when the researcher isn’t there.

Mass ethnography, whilst arguably methodologically superior, quickly reaches its limitations – time and cost are invariably prohibitive.

Mobile qual. goes the final 3 – 4 yards, to quote Steve, it “puts us into people’s back pockets”.

Face-to-face qual., by contrast, has systematic limitations:

  • momentary and thereby limited to point-in-time snap-shots
  • reliance on memory
  • empathy gap (an inability to imagine our “hot state” reactions when in a “cold state” of non-arousal)

Revelation summarizes how they see the advantages of online mobile qualitative:

  • ongoing dialogue (as opposed to a point-in-time snapshot)
  • immediacy: participants can record meaningful moments as they happen
  • Capture reactions more vividly, minimizing the process of distortion through rationalization. Mobile can get closer to System 1 type reactions.
  • easily-executed self-recording
  • ability to show not just tell (through use of photos, videos)
  • Directly access areas of the home that a laptop wouldn’t easily get to.
  • cost and reach (geographic restrictions are overcome,  travel is eliminated, saving both out-of-pocket and opportunity costs)
  • non-intrusive

Researchers can observe participants’ behaviors in more detail and more frequently, providing overall a higher level of granularity. Asking the question “why” is practically not limited, can be focused on behaviors that are particularly relevant to a brief, or that are not clear to the Researcher.

Mobile Qual. – Re-Evaluating Methodology

Steve is clear that he sees mobile as a medium, a means to an end – it’s not a goal in itself, nor is it new methodology.

Methodology could and should adapt, however, to what the new vehicle can to, its functionalities, to maximize its value and potential. The data-collection mode needs to adapt.

This means respecting and utilizing the medium’s versatility – especially true for smartphones.  These devices profiting from the convergence of Telecommunications, Digital and Entertainment technologies; one small phone can take photos, record sounds, transfer messages, phone calls….the list is long, if not endless.

Good mobile qual. research design means mimicking mobile usage habits, creating a design to suit that. Revelation refer to this as “participant-centric”.

It means moving on from legacy approaches.

Relying on a Q&A approach, however skilfully executed, is likely inadequate – mobile is a playful medium, capturing people’s imagination and holding their attention is key. New approaches need to be more involving, gamified, and above all – in Revelation’s view – activity-based.

Steve gives the following example to illustrate.

If a Client wishes to know the contents of a respondent’s fridge, it is a less enriching to say “Tell me what’s in your fridge”, far better to say “Give me a video-tour of the fridge”.

Video footage shows better than what a respondent can say in words – how tidy the fridge is, how well-stocked, if labelled, if certain people have sections, not just simply an enumeration of what is there.

Framing is also a key aspect to help maximize success with the mobile medium.

If a client wishes to understand how a certain audience enjoys cake eating, Steve suggests that it’s better to create a contextually focused challenge.

This could be framed as:  “Think about cake moments – which we’ll define as any moment you have cake, or any time when cake makes a moment better”. This would be preferable to simply asking: “what do you like about cake?” The “cake moments” approach results in a “Moments diary”, full of rich detail on the whens, the whats, the who-withs, the what-withs……a richly divergent process full of associative detail that a skilled qualitative practitioner can assemble meaningfully.


This leads to the center of what Revelation refer to as “Online Immersive Qualitative”.

Mobile – Closer to Experiences

Mobile is an immediate and quasi-omnipresent medium, with the ability to capture and transmit pictures, texts, impressions, feelings, behaviors as they occur. 

Mobile qualitative can profit from this by adopting what Revelation describes this as “immersive online qualitative”. It encompasses three core dimensions that help understand behavior:

  • Contexts: where are you? Who are you with?
  • Behavior: what are you doing?
  • Emotion: how do you feel?

Mobile can capture this triad of behavioral understanding when a particular experience occurs, making it much more likely to be an authentic and accurate record of an event.

Optimized mobile = imaginative technology + creativity in research

The learnings gained from the various validation and piloting studies reached critical mass in 2007, the year of launch. Revelation accompanied the launch with the announcement of their own mobile qual. App.

The App has evolved to third or fourth generation, but the principles applied are constant – “merge imagination in technology with creativity in research” – so the experience is smooth and engaging.

Some of the stand-out features are as follows:

  • Mimics mainstream current Social Media user experience - the App feels like Instagram, is visually driven, with room for text comment
  • Works off-line – ensuring thoughts or comments are not lost if connectivity breaks down.
  • Superior video and photo handling. The compression offered allows longer videos to be made, they are also easier and faster to upload. Videos can be uploaded in the background with no interruption of other device activity.
  • Push notification. Participants can be pinged a reminder or request by a Researcher who has seen something posted that is particularly interesting or pertinent, asking for more detail or clarification.
  • Device optimization. The interface is highly responsive and adaptive to whatever the device may be, re-sizing and re-visualizing automatically.

For What types of Research?

Steve named the following research areas as particularly appropriate for mobile qual.

  • In-store: respondents can take a picture, make a video
  • Online communities with diary-style activities: participant recording gives greater detail, richer contextual understanding
  • Outdoor activities – anywhere where mobile is at hand, overcoming recall issues associated with capturing later on a laptop.

Revelation see mobile qual. as particularly useful in what Steve refers to as “foundational studies” – where clients are going back to basics, to the fundamentals of what motivates and moves, often against a backdrop of wealth of existing quantified data. Mobile qual. is used to unlock, unleash, bring alive, develop an engagement strategy – move from a static to a dynamic insights approach.

An example: a customer has existing segmentation data, typologies have been identified, but there are questions on how best to engage key segments? How approach them? In this context of segmentation and typologies, mobile qual. works well in bringing customer types to life.

Case Study: Digital Dads

digital dads

In 2010 Yahoo wished to better understand the changing behavior of American “Dads” in the aftermath of the 2008/9 financial crisis. Some of these Dads had recently experienced being laid off, so had begun to assume a different role at home. Yahoo wished to know if, how and to what extent household tasks – cleaning, shopping, looking after the kids – typically assumed to be the role of the “Mom” – were affected by this dynamic.

Were some modern Dads being overlooked by brands and marketing as at least joint decision makers? 

The research design was mixed-methodological – qual/ quant., using online diaries and mobile qualitative.

The mobile qual. piece was chosen for the immediacy offered in low-interest categories – shopping for household goods, for example, or carrying out chores about the house. Activities that are easily forgotten. The design looked not just at Dads, but also recruited their families and their network of friends.

The findings suggested that men indeed were being overlooked by advertisers, sometimes portrayed as people unable to do simple chores.

Mobile qual. delivered authenticity - in-store reactions in particular – that brought quant. findings to life, adding a level of veracity and persuasiveness.

Case Study: USA Latinos and Hair Care


Another challenge posed to Revelation was in the area of hair care amongst the population of Latinos living and working in the USA. There are currently 52 million of them, they represent a fast growing opportunity as wealth levels rise, with a new and growing Latino middle-class segment emerging.

The client in question – P&G – could see a massive market and wished to gain a cultural hair-care perspective, understand how best to tap into this audience: what were their hair needs, how did they differ if at all, what products and brands did they know and use, which did they aspire to?

Smartphones were known to be the medium of choice for this audience, with many mobile-only households.

Mobile qual. was an obvious research approach suited to the brief – foundational insights - because mobile was a medium the audience would more easily engage with.

The design adopted was both fast – with a 3 day fieldwork period – and immersive. 20 Latinos were recruited and asked to perform exercises designed to understand their personal concepts of health and beauty. Using text and uploaded images, they were asked to use analogies and metaphors incorporating their five senses.

P&G gained an extremely rich, quasi real-time and cultural picture of what healthy and unhealthy hair meant. The insights were of clear value to the company’s R&D efforts.

Comment/ Outlook

  • Mobile qualitative research offers the ability to deliver the pictures, reactions, words of experiences as they happen. Often meaningful moments occur when the Researcher isn’t present – mobile can help overcome that.
  • Mobile qual. is well placed to provide an “aha” moment that a quantitative survey and arguably Group discussions can’t. It illuminates in a unique way.
  • Mobile qual. complements other research forms – it does things, gets to places, that traditional qual. doesn’t or can’t. Its delivers particular value in immersive type insights studies.
  • It takes Researchers into areas – the shopping aisle, the kitchen, the pub or restaurant – where they traditionally haven’t been, and where memory often plays tricks when respondents relate based on memory.
  • The method also allows a social and contextual component to be built in more easily – responses are given with the sense of place, occasion, atmosphere, and to what extent other people were part of an experience.
  • As part of the modern Researchers’ multi-modal armory, mobile qual. seems invaluable – relatively quick, authentic, cost-effective.

Jeffrey Henning’s #MRX Top 10: Bold Experiments in a Multi-Screen World

Of the 2,271 unique links shared by the #MRX community the past two weeks, here are 10 of the most retweeted.



By Jeffrey Henning

Of the 2,271 unique links shared by the #MRX community the past two weeks, here are 10 of the most retweeted.

1. Six Lessons From The MRS IMPACT 2014 Conference – Tom Ewing of Brain Juicer shares the six best ideas he took away from the Market Research Society’s annual conference: 1. Always be testing, 2. Fund bold experiments, 3. Beware stories and their limits, 4. Trust your actual stakeholders, 5. Use influencers to recruit additional hard-to-reach research participants, and 6. Diversify the ranks of researchers.

2. Market Research Debunked – Writing for RW Connect, Martina Olbertová debunks 5 myths: 1. Research is a data report, 2. Research is a substitute for lack of vision, 3. Answers are locked in the heads of consumers, 4. Research is only good for validation, 5. Research is purely analytical.

3. Ipsos European Pulse – An Ipsos survey of 7,000 Europeans in 9 countries reveals that citizens foresee a rise in anti-European movements in upcoming elections, while personally preferring that their country remains in the European Union, especially if the EU’s powers are reduced.

4. Google Flu Trends Gets It Wrong Three Years Running – The Google flu tracker has overestimated the incidence of flu for the last three years, according to David Lazar, of Northeastern University, who argues that adjusting the weighting can improve its accuracy.

5. Top 10 Key Mobile Facts For Market Research – Edward Appleton has compiled ten statistics about mobile device usage, from around the world. Key surprise to me: “The average level of mobile device ownership in 21 countries across the globe was 87% as measured in a 2012 Pew Global Attitudes Survey. The highest level of penetration was 94% (Jordan), the lowest Pakistan (52%).”

6. ESOMAR CEE Forum Bucharest 2014 – Betty Adamou recaps the keynote from ESOMAR’s Central/Eastern Europe conference in Romania (which promoters tout as “the Silicon Valley of Europe”).

7. Beauty Industry Robust Despite Slowdown – Euromonitor shares an infographic on changes in the global beauty market, including continued growth in the Middle East and Africa.

8. Spring into Action: Optimizing Tomorrow’s Market Research Effectiveness – The New England MRA has a call for speakers for its one-day conference in Waltham, Massachusetts, in May.

9. Half of Social Media Activity While Watching TV Relates to TV, Says Study – The Council for Research Excellence conducted a mobile diary study of 1,665 15 to 54 year olds and found that one in six times they watch TV they use social media and social media is twice as effective for attracting viewers to new shows as to returning shows.

10. How to Advertise in a Multiscreen World Where Mobile is the “First Screen” – Millward Brown conducted research in 30 countries to better understand multiscreen use. Consumers watch 7 hours of media today, with smart phones the primary screen in many countries, taking an average of 2.5 hours a day.


Note: This list is ordered by the relative measure of each link’s influence in the first week it debuted in the weekly Top 5. A link’s influence is a tally of the influence of each Twitter user who shared the link and tagged it #MRX.


Cat Polling To Replace Traditional Political Polling

Posted by Brian Singh Tuesday, April 1, 2014, 0:13 am

With remarkably low response rates to telephone and online panel surveys at this time, Canadian polling & market research agency Zinc Tank is pleased to announce the launch of its Pet Electoral Tracking Study (PETS).



By Brian Singh

After a host of polling issues in the 2012 elections, today there is a real option in more accurate public opinion research. Cat polling. But not only cats. All pets.

With remarkably low response rates to telephone and online panel surveys at this time, Canadian polling & market research agency Zinc Tank is pleased to announce the launch of its Pet Electoral Tracking Study (PETS).

With approximately 80 million pets (2012) in the United States, it’s no surprise that owner’s love to speak on behalf of their cats and dogs. The Zinc Tank research team has tapped into this passion and developed the proprietary neuroscience technique Anthro Linguistic Projective Organization (ALPO). This has led to remarkably incredible response rates and more representative data than has been seen for years. Zinc Tank found that cat surveys are highly accurate because cats are highly random – a 1,000 cat poll is by definition a “random” sample. Such a sample constitutes a herd, and “herd” is the new listening – the successor to social media monitoring.

“We have found that owners considered their pets politically engaged and accurately reflect the opinion of the household,” stated Chief Methodologist, Brian F. Singh. “An intriguing finding was that grey cats tended to vote Republican, while black & white short hairs leaned Democrat. Siamese were almost exclusively Independent.”

“As a critic of all traditional polling methods, we feel that cats and dogs opinions best represent the state of party preference and performance of leaders. PETS is proven out-of-the-litter box thinking. With its success, and US Census Bureau’s cat census notoriously out of date, we are in discussion with Google to develop “Google Cat,” to help calibrate their insights algorithms.”

President Obama’s dogs, Bo and Sunny, were unavailable for comment. But rumor has it they are independent voters.

I think we can all look forward to PETS playing a key role in the 2014 mid-term and 2016 election cycles.

We’ll be showcasing this amazing new innovation at the 1st ever IIeX Animal Innovation Forum in collaboration with the ASPCA and American Kennel Club at our event in Atlanta in June.  Stay tuned for more details!



*Note: This is an April Fool’s prank. With the exception of the opening statement, none of this is true. Polling is an important part of living in a democracy, but we can also have some fun at its expense. We hope that this parody made your day a little more entertaining. No animals were hurt in the preparation of this news release.

From the Client Side: Interview with Stacey Symonds, Senior Director of Consumer Insights for Orbitz Worldwide

To hear more from the perspective of the corporate researcher, this occasional Greenbookblog.org feature will spend some time with a researcher From the Client Side. This fifth interview continues the discussion about Next Generation insights techniques and the future of the consumer insights industry with Stacey Symonds of Orbitz, Worldwide.

client side


Editor’s Note: In preparation for the last GRIT report Ron Sellers conducted a series of IDIs with client-side MR professionals. Some of those have been turned into interviews for this blog series on client-side views, and today’s is another amazing example of the very pragmatic and progressive view of the evolution of the role and process of strategic insights espoused by many client leaders. Stacey Symonds offers a very grounded and reasoned view into how Orbitz is embracing the best of the industry, both “traditional” and “emerging” and it should be required reading for all suppliers.

Stacey will be one of over 40 clients joining us on stage at IIeX North America in June and I for one can’t wait to chat with her more in person!


By Ron Sellers

This interview features thoughts and observations from Stacey Symonds.  Stacey is Senior Director of Consumer Insights for Orbitz Worldwide.  In her current role, she partners with a range of internal business leaders to integrate the voice of the customer into day-to-day as well as long-term development.  She has over 15 years of experience in client-side customer insights, brand strategy, and market analysis in the automotive, retail, financial services, and travel industries.  Stacey holds an M.A. in Applied Social Research from the University of Michigan and a B.A. in Textile/Apparel Management from Cornell University. She serves on the External Advisory Board of the A. C. Nielsen Center for Marketing Research at the University of Wisconsin-Madison School of Business, and lives in Madison, WI with her husband and two energetic kids.

Ron:  First of all, what research or consumer insights methodologies or approaches have you used over the past 12 months or so?

Stacey:  Well, it’s been a very wide range because in my role I have responsibility for all consumer insights at Orbitz, so that can range from brand tracking to product development where we’re doing some things like workshops with consumers to do some co-creation, to online discussions, to bulletin board discussions.  So really it runs the gamut.  I would say I skew toward quantitative studies, though, like discreet choice and other trade-off modes.

Ron:  Are there any methodologies or approaches you’ve intentionally stopped using, or even considering, over the past 12 months or so?

Stacey:  I’m not a huge fan of traditional focus groups with the one-way-glass kind of interactions.  I just think there’s too much of a group dynamic that goes on.  There are very specific cases where they might be okay to use, and I rarely have those cases.  I need decision-making data.  I don’t feel like I can get it from that mode.  I’ve used them more in other places I’ve worked, but here that’s something I definitely don’t tend to use anymore.

Ron:  Is that something you’ve become more uncomfortable with because of some of the new techniques that are out there, or is that totally independent of these new techniques? 

Stacey:  It’s actually independent of the new techniques.  For me, it’s more that I think we were using a screwdriver to hammer in a nail.  When I first got to Orbitz a little over three years ago, they were using focus groups for concept evaluation and progression.  When you just don’t get a definitive sense of which concepts should move forward, it becomes very subject to interpretation, versus doing a quant study where you can actually do some rotation and get some clarity on what is working.  Actually, to your point, though, one thing we are doing is a qual/quant hybrid, which takes the place of a focus group.  It gives you a larger sample size and then also lets you explore language and how people are feeling about things, so you kind of get both sides.  And in that format, too, the consumers are not sitting in a room, but they’re virtual.  So you don’t get that group influence unless you want it to come into play.  You can control it, versus in a focus group were you don’t have control over it, and it’s a small sample size, where you just get this influence that you may or may not want in your decision making.

Ron:  With all the new consumer insights methods that have emerged over the past five years or so, where do you see the market research or consumer insights industry headed in the next few years? 

Stacey:  I think there definitely is a move to quantitative.  I’ve seen that in a variety of forms, even taking data that might previously have been called qualitative and making it quantitative.  Text analytics is a good example.  I think that’s definitely moving into a more mainstream space.

The market research that I have seen used most is information that helps you make decisions. And I feel like quantitative helps you do that in a more deliberate and consistent way.  So I think that’s been an evolution for sure.  Very fast-turnaround, self-service methodologies are here to stay.  I don’t think they’re going anywhere.

I think there may be some discussion, too, about the evolution of panels in communities.  It’s something I’ve kind of struggled with – where I fit on that continuum.  Because I feel like when you empanel somebody, either for just a generic research panel or for an online-community-type engagement, you change something.  The moment they become a part of that community, you change how they feel about the company and how they answer questions, so I don’t know how that’s going to evolve, but I bet it will in some meaningful way.  Either we have to just acknowledge that and just keep using it because it’s convenient and easy, or we find other ways to do it, like what I think Google’s consumer surveys is doing in going out to a broader audience where they can actually get to people that you may not have in other forums.  You may get a rawer, more honest perspective on things when you do that, versus going to a friendly kind of a panel environment.

Ron:  It’s interesting – you’re saying you see more of a move to quantitative, and of course the example you gave was something that’s like big data in that it’s existing information that people are now quantifying.  But what about the traditional quantitative surveys, whether using an online access panel, a panel you create, a phone survey, even a mail survey?  You’re saying there’s a move to quantitative data, but at the same time there are increasing concerns about the representativeness of traditional quantitative methodologies.  I think it was Pew that recently estimated that the typical phone response rate now is nine percent.  It’s not random-probability sampling when you’re using an online panel.  So how do you deal with the lack of representativeness, even as you’re saying more and more you do quantitative? 

Stacey:  First it’s recognizing that issue where it exists and making decisions given that fact.  So I caveat the heck out of things now to make sure people understand.  There’s always been a say/do gap anyway.  So even if it was a completely unbiased perspective, what somebody tells you is not always what they are going to go do.

For one thing, I argue that we should use multiple methods, and we do.  It’s observational plus it’s surveying.  But the other thing is I do feel like we potentially need to adapt how we ask questions, and therefore how surveys are done.  The method might be still useful where you can focus a person’s attention on something for a period of time, whereas if you’re in social media, you have no control over what they talk about, how long it gets talked about, how deep they go, or how much they understand it.

I think there’s still a role for the survey in the world today.  But I do think they need to be made more engaging so that a broader cross section of people will want to do them.  So instead of sitting there and saying, “Gosh, I have to sit here and answer 30 questions, text question after text question,” we have more visually engaging, more interactive methods I think could help counter some of the trends we’re seeing here and still give us useful output.

Ron:  Where and how do you learn about new research methods and new approaches? 

Stacey:  Some of it is definitely going to conferences.  I think attending a couple of major market-research-focused or advertising-research-focused conferences definitely helps a lot.  And definitely scanning publications that are out there, whether GreenBook or Quirk’s, is a really good resource as well.  Then just the grapevine.  Certainly, the first time I ever heard about Google Surveys was somebody at work who’s actually not even a researcher.  They happened upon it when they were surfing the Internet.

Ron:  When evaluating an approach you haven’t used before, what are the factors you look at to determine whether it’s something you believe is valid or something that you kick to the curb because you just don’t think it’s usable? 

Stacey:  Well, a couple things for me.  One, I try to avoid the “shiny new object” syndrome.  Just because something is new doesn’t mean we should go do it.  And I think partially that’s because I’m not dealing with a huge budget.  I think even if I did have a huge budget I’d want to use it very judiciously.  So I don’t try things out just to try them.  I actually want to be fairly educated first to make sure it’s something that could really fill a gap in what we know.

The other is actually more of a cultural reason than anything else.  I want to make sure I’m in tune with what my organization needs and can tolerate, in terms of risk and acceptance.  I am in a very data-driven organization, but consumer insights is a new practice for them over the past couple of years.  So if I went in and said, “Okay, here’s this new, really complicated technique; we’re going to try this,” I think they might be open to it, but whether or not I could really sell it in…  I think it would be harder to do if I didn’t feel confident that it really was something I knew could add value.  So I’m not so speculative about most things.

Ron:  The industry obviously has a lot of what I’ll call the traditional approaches, such as IDIs or focus groups, intercept interviews, ethnography, and all the different survey methods.  Which of these, if any, do you feel are still valid and useful today for your work?  And which them also do you feel will be valid and useful five or ten years from now?

Stacey:  I would say definitely the face-to-face research and survey techniques are just moving online.  I think we still use them, but we’re doing it in a way that’s more flexible for the person involved.  We have done group discussions by webcam.  We’ve used it for ethnography.  Where you want to go really in depth with somebody, I still think qualitative questioning is one of the best ways to do that.  We just might do it through an online bulletin board exchange, an online diary, or something like that, instead of sitting in a room and doing a one-on-one interview.  That I think has changed for sure.  And going forward I wouldn’t expect that to revert back.  That’s just sort of how I feel about focus groups and where I think those are.

I think the idea of co-creation or discussions with consumers, there’s something to that where you can do it either in person or virtually where you can make it less subject to group think, depending on how you structure it.  So that’s definitely something I feel like is evolving into just a little bit of a different space.

Ethnography, I still feel like that’s a great tool. And I think that approach is one that also is evolving with technology.  So ethnography now is:  How do people use mobile phones?  How are they using websites?  All of those things.  I think it’s the same principles; it’s just being used in a different channel.

Ron:  You’re talking about how a lot of different things are moving from in-person to online.  How much of that do you think is driven by, or unique to, the fact that the customer interaction with a company like Orbitz is online?  Shoppers are not going to the Orbitz store or the Orbitz dealership or taking a package of Orbitz home with them.  Do you think it’s influenced partly by the type of company you work in?  Or do you feel like if you were working at Post Foods or Walmart you’d have a lot of that same perspective? 

Stacey:  I think I would still have a perspective that digital is a way to do research that can get you past geographical issues.  It can get you to a consumer who couldn’t show up to a group discussion or to an individual interview.  You can get to them.  So I feel like it’s an access issue.  It is true, if I worked for Walmart, I would be in stores and doing things there which might involve talking to people in person, in a store.   So there’s some of that.  But I do think still the digital piece is something that lets us do more with less as budgets have gotten squeezed.  It also has that added benefit that it just takes some of these barriers away that we’ve had before that really might’ve narrowed our ability to reach different types of consumers.

Ron:  We talked about both traditional and non-traditional research approaches, but what about some of the stuff that’s a little bit more out there; a little newer?  Eye tracking, facial analysis, mobile MR? 

Stacey:  I’m involved with a project from the ARF (Advertising Research Foundation) which tested ads with various methods of neuromarketing research and biometrics including EEG, Skin conductance, fMRI, and eye tracking along with traditional survey techniques.  It’s almost like a bake-off to figure out which of those is better.

I think the cost/benefit equation is still difficult for any of the really new techniques, especially neuro- or bio-related. These methods do tell you if someone has a reaction to something, but it is not always clear if it is positive or negative reaction and how that might impact the effectiveness of your advertising. Eye tracking has been around the longest, and it does tell you how much time someone spends looking at something so you can optimize your use of space and visuals more so than with other methods.

Ron:  What about facial analysis or facial coding?  Do you have any experience with or thoughts on that? 

Stacey:  No.  That was not part of this study, but we are actually using for the first time now.  It’s the same thing as with those other techniques.  They’re so subject to interpretation.  Yes, there are principles and some things that people have observed over time, but I am still not sure I could go to my CEO and say, “Hey, because this person has this facial expression, we should go do something different.”  That said, we may be able to use facial coding to uncover where people say they have a negative stated reaction to something (like more risqué humor), vs what they actually find amusing.’

Ron:  Those are all methods that, to some extent, are used in partnership with traditional research approaches.  Then there are the completely new approaches that are either totally separate from traditional research techniques or some people see as a replacement, which would be behavioral economics, big data, social media monitoring, and neuroscience.  Which of these do you feel like are valid and useful today, or will be very soon? 

Stacey:  I think social media analytics can be overrated depending on the industry you are in and your context.  This is a big disclaimer.  We’re a price-driven industry. We don’t have a lot of people out there debating the benefits of Orbitz versus Expedia, in general, or at least it is not done in public.  The discourse is more, “Here, I got a promotion code for Orbitz; do you want to use it?”. We do use it for customer service issues or getting reaction to new campaigns.  In other cases though, the dialogue is typically not meaningful enough for us to use it in a robust way for generating insight.

Now, if you are like a P&G, for example, I could see where that could be more useful to you to see how products are being discussed and uncover issues.  But, again, you don’t control the conversation; you’re just observing it and you’re just trying to figure out which takes a tremendous amount of time to filter through  what are people talking about?  And it’s probably 1% valuable, 99% not.  And the time and effort it takes you to get to that 1%, my sense has been I don’t know if it’s worth it.  But again, it could be different in different industries, so I think that one is probably context-dependent.

I think mobile research was one of the other approaches you mentioned.  That one also depends on what business you’re in.  For us, that’s actually just part of my normal mix of tools I use, because we do have mobile apps and that’s very important to our business.  That, to me, is just a channel.  It’s not really a new technique, per se.  It’s still a survey; it’s just on a smaller device and has different usability.  But it’s still a survey at the end of the day.

Ron:  What about big data? 

Stacey:  Big data, to me, can take a lot of forms.  I like to use it in a very focused way.  So I make sure I triangulate, because you can’t trust any one data source these days; you need to have several to be able to really understand what’s going on.

For example, if I want to understand why people are canceling hotels, I want to look at all the people in our big data who have canceled a hotel.  How far ahead of time did they cancel it, and how did they cancel it?  Was it on a phone?  Was it on a desktop?  And then also do a survey of people who canceled and ask them why they did it.  So I feel like using big data in focused ways is meaningful.

It certainly can be valuable to have a complete and accurate record of things, because everything’s in there for click-stream data, for example.  If we want to understand how many people are leaving our site after visiting a certain page, we have that data, and we have to go and query and find it.  And I think actually those front-end tools that sit on top of big data are where the value is.  If you can get good tools to do that and help you quickly get to answers, that’s when big data also becomes more useful, is when your access points really enable you to surgically go into it versus trying to deal with all of it.  I think that’s maybe where there’s more value as well.

I think just complete, open data mining – potentially you can learn some things, but you have to have the resources to be able to do that.  I think it’s hard for a lot of companies to have somebody whose job it is to explore without knowing what they might find.  That’s just something we don’t really have the luxury of doing.  But if other companies do, that’s great.  I just think it’s a huge amount of effort to uncover something without having some type of business question you’re trying to answer with it.

These companies that came in and started with big data figured out there was something missing, and that was, I think, the why, and that’s where consumer insights comes in.  Why are people doing this?  Why do we see these patterns of behavior?  And it’s become a part of their decision making.

Ron:  Some of these new approaches require fairly specialized skills that a lot of traditional researchers don’t necessarily have.  Do any of these tend to make you nervous about your own skill set or future in the industry? 

Stacey:  I would say not really.  I have sort of an eclectic background anyway.  I have a master’s in applied social research, which was sort of a multidisciplinary program at the University of Michigan, and it has a lot of different aspects to it.  It has statistics, business social psychology and it had research methods and the cognitive psychology of asking questions.  So I feel like I actually had a very good background that enables me to go in different directions.  It enables me to go to the quant stuff.  It enables me to go to behavioral economics.  I would say I don’t feel too worried.

But I do think that if you want to go into research today, you should have a pretty broad base to draw on.  If you just learn traditional survey questioning and not the digital world of web analytics and big data, that will probably set you up for failure.  It’ll be much harder for you to figure out what’s going on.

I do think, too, there is a flexibility gap sometimes.  I find research can be a very logical, process-oriented activity, but I think there’s actually a need for more rapid thinking and flexibility in the field.   It may be harder for some researchers to move there, especially if you’ve spent your whole life doing a certain type of research and a certain function, especially in a bigger company.  I think it’s harder to adapt after you have been in that place.  It certainly depends where you are, too.  I happen to be in a job where I have to know all these techniques and use them. That’s just part of my job.  For me, I feel confident I can move between methods pretty well.

Ron:  When you consider all these different new techniques and approaches, which of these do you feel tends to be more true about our industry today:  A) Too many research professionals are dragging their feet and need to get on board with these new approaches, or they’re simply going to be left behind; or B) Too many research professionals are abandoning proven methods and jumping on the bandwagon of these new approaches without sufficient proof that they’re valid or meaningful? 

Stacey:  I think I’m going to be totally in the middle.  I think I’ve seen some of both.  And I don’t know which I agree with more.  I think there’s a little bit of each of them.  I don’t think we should forget about what’s good about traditional research.  I feel like sometimes principles are disappearing, like how to ask questions well.  That means we’re not asking things in the most effective ways.

I think some of the move to new methods is also causing people to focus less on the fundamentals.  That is an issue.  On the other hand, these new methods do represent where we’re going.  And I think if you want to be successful going forward, especially in today’s world, where people do have to move between companies or between industries, if you’re not open to some of these new things, I think it will be difficult to keep up.

Some of the core suppliers – some of the biggest suppliers in the world – still do rely on some of the really traditional techniques and bear a significant cost structure because of it.  And I just think they’re also probably going to have to evolve a bit in order to be successful in the long term.

Ron:  When you deal with people who say things such as, “In-person qualitative is dead,” or “Survey research is obsolete,” how do you respond to those folks? 

Stacey:  I feel like nothing is absolute in this world.  At the end of the day, there are different applications where different methods make a lot of sense.  And I think the industry might also be different.  But I tell people, at least for us, it’s like being a reporter.  If you don’t understand all the W’s and the how about something, then you don’t really understand it.  You can’t get the what, where, why, who and how from just observing somebody do something, just from social media analytics or just from click-stream data.  You need to get the why.  And sometimes the only way you can get a why is to ask somebody, or you could just decide, “Okay, I’m just not going to know it.”  And then you’ll be less educated for it.

It’s not an all-or-nothing kind of a thing.  I think there are benefits in these things; these techniques.  There are benefits in focusing a discussion on a topic that somebody may not otherwise think about.  I just think to help businesses grow, we’re going to have to use the best of what’s out there in order to really be successful.  And some of it is going to be more traditional and some of it’s going to be new.  I think that’s where the good intersection of this industry is.

Ron:  One thing I have seen with a lot of these newer approaches is that it’s not just, “Here’s a new tool,” but it’s, “Oh, you do this and you can replace this other thing.”  For example, “If you do social media monitoring, you can replace your surveys.  You don’t ever have to do a survey again.”  What does your reaction tend to be when you see that message coming from vendors offering these new tools?

Stacey:  You know what?  I actually think it erodes their credibility with me personally to say there are these absolutes out there.  At the end of the day, it’s sales, so I am pretty skeptical of that.  I need to have things be more proven before I’ll buy into them.  And I would never buy into that idea that there’s one solution to everything.  It doesn’t make sense to me logically, just given my experience.

Ron:  With some of these newer techniques that you’re using or you would consider using, are you looking to your traditional research vendors; vendors you have used for traditional research in the past?  Are you looking for them to work with you on those approaches, or are you looking for new vendors that specialize in the newer methods? 

Stacey:  I think it is a little of both.  It is difficult and disruptive to change vendors all the time, especially if you have brand tracking, ad testing, or other things like that.  On the other hand, I kind of watch.  I kind of plant some seeds and I watch and see if the companies I’m working with are adaptive or not.  I’ve left working with some companies because they do not seem to be adapting.  For me, it’s also about the relationship.

But I feel there are some vendor partners I work with that have clearly evolved, and I’ve worked with them for more than a decade now because they evolved and are proactive about my business needs in a really positive way, and others are kind of left behind because I feel like they’re not evolving, or the way they’re evolving is not as value-added for me.  If they’re not moving into these new spaces, in a way, then I feel like it’s sometimes easier to do things myself than it is for me to go through a supplier who does it for me.  So it’s trying to find that balance.


How TNS Is Validating Mobile Globally

TNS is taking the lead with a serious look at mobile in market research as a global opportunity.



Editor’s Note: Edward Appleton is doing a series of posts focused on the client-side view of mobile research, with an emphasis on use cases and best practices learned so far. This is the fourth  post in that series that we’ll be publishing over the rest of the month. Parts 1 – 3 can be found here.


By Edward Appleton

TNS is one of the leading global Market Research agencies, with operations in over 80 countries, and part of the Kantar Group – a leading provider of insights and intelligence. Millward Brown, the Futures Company, The Added Value Company, Kantar Worldpanel, Kantar Retail and Kantar Media all form part of the group.

TNS is taking the lead with a serious look at mobile in market research as a global opportunity.

It was the main sponsor at the recent MRMW Conference in London (Market Research in the Mobile World – 10/2013 – http://bit.ly/1aeYQ6d); they have presented papers and case studies at various industry Mobile Events over the past 2 years.

More importantly, they are building an evidence-based approach to understanding mobile research. They execute annual large-scale quantitative Global studies on mobile  – TNS Mobile Life (http://bit.ly/1bloPNd)  – covering 43 countries, interviewing just under 38.000 mobile users globally. They are beginning to document the pros and cons of the different types of mobile surveys (Apps, WAP, SMS, USSD).

This empirical approach of global exploration and validation is extremely beneficial to the research industry as a whole, as it will create confidence amongst Client side researchers.

Their approach extends to a healthy skeptical attitude to what they see as massive hype surrounding mobile as a panacea to all marketing ills.

A recent blog on the ESOMAR site by Sam Curtis (http://bit.ly/1etPZFf), provocatively entitled “The Big Mobile Lie”,  shared evidence from TNS studies showing mobile devices being hardly used by shoppers at the point of purchase once usage data is broken down to category level – including pet food, alcohol, tobacco, OTC medicines.

So: what’s hype, what’s “for real”? What does TNS’ empirical approach currently reveal about how to best approach mobile research – for what types of study, how best design, what caveats?

I caught up with Sam to understand TNS’ views better.

Mobile: Hype or Seachange?

TNS actually shares much of the underlying excitement for mobile witnessed by many media and marketing organisations.

With the rise of Smartphones, the potential of mobile to change many industries radically is real –  retail (“showrooming”), finance (mobile banking, mobile wallets), health (m-health)  are examples where swift change in value chains is already happening.

Smart phones transform the device into something completely integrated, offering all sorts of experiences in one go. It offers consumers superior convenience and flexibility to explore, interact, buy – wherever they are, when they want, whilst at the same time offering entertainment (music, videos, games) connection to friends via social media, and the opportunity to keep up to date on events.

My sense is that TNS sees both the excitement and disruptive threat of mobile – something that will likely destroy traditional business models – and wish to be position themselves as the go-to insights consultancy offering marketing business advice on a range of issues beyond research.

Significant investment and senior talent within the Kantar group is being focused on understanding how best to use mobile in research, according to Sam.

In the research world, they estimate that within 2 – 3 years between 20 – 30% of all data collection will be mobile, a massive shift from today’s lower levels.

Equally, they see little evidence yet of a mobile “shopper revolution” – many FMCG items are frequent purchases, habit-driven, with little use of the mobile to check prices or find out more about the brand at the POS.

So – mobile is a sea change, but with significant dangers of unsubstantiated claims – hype – leading marketers to potentially invest indiscriminately into mobile as the hot topic of the moment.

Forces Driving Change

For research, TNS sees the following issues and trends that make them view mobile data collection as a strategically important topic:

  • Emerging markets will go mobile first, skipping the phase of laptop/desktop usage which characterizes many developed markets. Emerging markets are huge growth opportunities for many major multinational companies. Missing out here on growth fueled by mobile isn’t an option.
  • Dropping engagement levels on online panels. Mobile is potentially an answer to what is a widely recognized (if poorly documented) issue - relatively high churn levels of people dropping off MR panels, with recruitment of new users proving increasingly difficult. TNS sees mobile as a way of attracting new, particularly younger audiences to participate in market research.
  • Recall gaps. Our memories are flawed; data sets relying on recall are often not complete and therefore lack accuracy and granularity. Mobile is an “in-the-moment” medium. Not only can we capture the “what” more accurately, but also the way we feel at that point in time. Mobile overcomes the recall gaps.

The most powerful driver to mobile, however, is one that is relevant to the research industry as a whole: the need to do shorter surveys.


Researchers have long been aware that conducting long surveys – arguably anything over 15 minutes – is in danger of being counter-productive: respondents’ engagement levels drop, their enthusiasm for future survey participation falls, and data quality suffers as a result.

Some of the key reasons for long surveys surviving in a world of ever-decreasing attention spans are as follows:

- Comparability: any change in methodology will likely result in the data changing. If you have tracker studies running for a number of years, with internally accepted KPIs, then switching data collection mode needs careful transition, calibration planning, and internal stakeholder sensitization. It is a bold but important call, that requires senior Insights staff that are both respected, well established and confident.

- Knowledge Needs: Clients (often Marketing staff who are the budget holders, but with little or no formal MR training) naturally wish to squeeze every last question in “once we have them there”. Micro-surveys are a relatively new phenomenon – Google Consumer Surveys was launched in 2011, and was limited initially to the US and Canada. The notion of “chunking” will be new to them – education is called for.

- Commercial forces: many Agencies  are reluctant to push-back on Survey length for fear of losing a piece of work. This is a spiral towards lower quality and needs to be broken out of, with the more commercially stable industry leaders playing a key role.

- More Complicated Scripting: scripting a mobile survey is no doubt more complex due to the myriad types of devices and operating systems that a given survey has to work with. This requires accessing sufficient staff with the relevant skill sets, investing in that resource. This can easily raise the base level cost of a business, which Agencies may or may not feel comfortable about passing on to clients.

The barriers are significant.

Mobile, however, has witnessed such rapid and widespread adoption amongst consumers worldwide that it represents an irresistible force that Companies – Client and Agency – will have to react to. The phase of “simply being prepared” is probably behind us, knowing how to integrate mobile into the research mix is an imperative.

If – as TNS suggests – mobile actually represents an opportunity by which the respondent experience for all types of MR engagement can be both improved and made more predictive, it has the potential to transform – revolutionize even – the way we conduct huge swathes of survey research.

Mobile: Shorter, with Higher Predictive Validity


TNS is currently conducting research-on-research studies in selected geographies and categories – focusing on brand equity, advertising tracking, customer satisfaction studies – to establish potential differences in response patterns between mobile and laptop/ desktop. They also are aiming to find questions that are the most predictive of actual behavior.

Key survey metrics – including Purchase Intent – are compared across the different devices to actual subsequent purchasing behavior. Analysis is at the more critical respondent level.

The topline meta- finding is startling: shorter, more relevant surveys have a higher predictive validity than longer ones.

This has major implications for interview length across survey type.

TNS can pinpoint which questions in a given survey type are more predictive, correlate well to actual behavior, and eliminate questions that are redundant.

Work TNS has conducted in 3 countries across two categories show the following questions to be redundant, as they have a low correlation with individual purchase behavior:

  • aided awareness
  • brand familiarity
  • brand satisfaction
  • purchase intent
  • recommendation (NPS)
  • brands bought – past 3 months, regularly, most often
  • brand attitudes (i.e. ‚this is a brand i trust‘)

Parallel to this work on mobile validation, they are working broadly with clients to implement “survey-length-reduction” principles:

  • Respondent Level Validity (only asking questions that respondents can answer accurately)
  • Principle of Redundancy (detecting respondent level correlations within-survey and eliminating unnecessary questions by using auto-fill)
  • Relevance (only asking questions about the few brands and attributes that participants really care about)

Brand and attribute lists are reduced radically, only one brand-rating question is asked per brand.

The resulting shorter survey (an equity study in this example) can be conducted in 3 minutes with a respondent-level validity of R = 0.62, and an impressive correlation to market share of R=0.90+.

Their conclusion: shorter surveys can give higher predictive validity.

Trackers and brand equity studies that could traditionally require 40 minutes of respondents’ time can be reduced to well under 10 minutes – as little as 3 in the case quoted.

This opens the door for many trackers to transition to mobile, saving time, eliminating respondent fatigue, reducing cost, and delivering better results.

The insight that shorter is more powerful is intuitive: we pay more attention if we are allowed to talk about the few things we care about in a category, and not get forced to answer multiple questions about things we may have no opinion about. This is still the case with many trackers.

It is as TNS states nothing less than a revolutionary approach to surveys – whatever data mode participants choose to respond.

In summary, best practice in mobile could drive best practice across mode type. There are probably few types of surveys that cannot, once condensed intelligently, using sufficient computer power and intelligent programming, be conducted on a mobile device.

Mobile: Increased Accuracy


Alongside forcing Survey designers to think carefully about asking fewer questions, mobile helps solve the known problem of memory gaps. It delivers more accuracy.

This is particularly relevant for diary-formats, but also for any out-of-home buying or consumption occasion.

Delivering better accuracy can also be challenging: the emerging picture can be radically different from one delivered by memory-reliant desktop/laptop.

The following case looks at how drinks are ordered in a pub, and shows how different key outputs can be.

Case Study: Molson Coors On-Trade Lager Drinking


TNS was asked by Molson Coors to take a quantitative look at drinking in pubs in the UK (the on-trade), which in their view was an area thin on robust insights.

Industry wisdom – and one shared by the UK Government – was that alcohol consumption is price-sensitive, and that raising prices leads to a fall in consumption. Molson Coors wished to validate this, understand better what the key reasons driving brand choice were, including price, as accurately as possible. Mobile seemed an ideal vehicle for research.

TNS used a split-design approach, comparing online laptop/desktop data collection to mobile. They recruited 147 lager drinkers in the UK; and asked them the same questions regarding their drinking habits at a recruitment stage, (using a laptop), then complete the same questions whilst in the pub on their mobile.

The key findings showed clearly that using mobile gives a different response:

  • 3.8 influence sources were mentioned in the recruitment survey – in mobile there were only 1.4
  • A sharp swap of priorities was noticeable: price and special offers were stated as the most important influences in the survey completed on a laptop/desktop, whereas the in-the-moment responses via mobile showed “brand” to be the most important factor.
  • Special offer hardly figured as an influencer in the mobile survey.

The two versions challenge received opinion.

Whilst the “truth” isn’t necessarily accessible via a single-mode research approach – no triangulation was undertaken or at least made public – price and promotions appear to be less important in marketing in the on-trade as a traditional approach suggested, whereas the role of the brand is up-weighted.

This has huge implications for the Molson Coors marketing mix – suggesting that brand-building activities for online preference are key, the marketing mix needs to respect this. Targeted TV advertising (one example) should be given serious consideration.

The conclusion from this case: mobile is immensely valuable in getting marketing a step closer to out of home experiences as they are felt and recorded at the time.

This greater accuracy can lead directly to:

  • more efficient marketing-mix
  • a high likelihood of an improved ROI.

Mobile – What Types of Research?

Sam sees two main areas where doing mobile research has clear advantages

  1. Path-to-purchase journeys
  1. Touchpoint analyses

i)  Path to purchase journeys


TNS’ global study on mobile usage – Mobile Life – contains data showing that at present, mobile is used relatively more extensively for in-store intelligence gathering than purchasing.

There are clear differences by level of market and mobile maturity, with retail mobile usage increasing with rising smartphone penetration levels.

In Europe, 38% of mobile users have used their device at some point to research an in-store purchase whilst only 16% have actually used their mobile to buy something. Younger respondents are more likely to browse on their mobile in-store. The use varies strongly by category – usage levels on an individual trip basis, as Sam points out in his RW Connect blog, are often very low. TNS refers to “channel not fulfilling potential”.

Overall, the data suggests that some browsing has shifted from in-home (desktop) to in-store (mobile).

Using mobile to understand better how consumers react to in-store promotions or shelf-displays is therefore of value; a need which is likely to increase.

Mobile research should be executed in categories where the extent of in-store mobile browsing merits attention – books, DVDs/games and computers top the list, albeit still at single digit penetration levels.

A further consideration for mobile path-to-purchase studies is length of purchase cycle.

Engagement levels and diary entry enthusiasm can be held high for a relatively short time period, typically up to 4 days. After this, interest levels and reporting intensity drop.

For categories with particularly long decision making cycles, mobile tracking is not yet easy to execute successfully in TNS’ experience. As mobile panels become larger and more robust, this is likely to become easier.

Mobile diaries work well by using “near-the-moment” self-reporting: respondents are asked 4 – 5 questions at the end of each day, until they make a purchase.

ii) Touchpoint analyses

brand touchpoints

Mobile offers the following advantages:

- all touchpoints are recorded

- event sequencing can be monitored

Respondents’ behavior can be linked to a touchpoint – so allowing researchers to see precisely what prompts (advertising) or interactions (advice, recommendations, expert opinion) lead to an event, a purchase act.

The journey can be mapped over time.

Mobile allows a better understanding of the context in which the decision was made, as the recording is much nearer to the time it happened, and less reliant on erroneous recall.

“Touchpoint correlation” can also be identified: patterns can be identified of where the use of two or more touchpoints occur in clusters frequently. This allows researchers to probe the causality, engage in the diagnostics of “can you tell us more about…..” as an improvement on the direct and often blunt technique of “why did you….?”

Mobile Surveys – Good Practice Tips


TNS talks of “good” rather than “best practice”, because:

  • the medium is still relatively new
  • insufficient R&D work has been done across audiences, geographies, categories

It is premature to issue a clear and comprehensive set of survey design guidelines.

Their overall comment on the state of mobile research is critical, that there is insufficient recognition that simply transferring an online survey onto mobile whole-scale isn’t the answer. To quote Sam: too many mobile surveys look like a regular survey.

Their current guidance on how to shape good respondent mobile survey experience is as follows, based on pilot studies completed between 2011 – 2013.

  • Short Interview length: maximum 10 minutes, but ideally much shorter, at 3 minutes.

This conflicts with all the knowledge needs a given project may have. It would require clients breaking up a large single surveys into multiple smaller ones. This means Agencies need to show Clients options that deliver on cost and timing – doing multiple shorter surveys instead of one longer ones.

This is an area that as yet is not standard practice in the development of proposals, and represents an opportunity for the industry. Education is needed.

  • Eliminate repetition

Questionnaires often – in TNS’ experience – cover the same areas twice, possibly more often. The drive to brevity requires any potentially overlapping questions to be merged.

For continuous surveys, this involves showing Client stakeholders relevant analysis (factor analysis, predictive analytics) and proof of redundancies, levels of inter-question correlation levels.

  • Filter for Relevance

Respondents should only evaluate brands that are in their consideration set. They don’t need to see a whole battery of say 8-10 brands or a list of 30 product attributes. This makes survey tasks – of image evaluation for example – much shorter, and manageable on a significantly smaller screen.

Filter questions are key: only ask questions on the attributes that respondents care about, for example, or the brands they have in their relevant set.

Grids are much simplified by following this guideline, and responses correlate more strongly with actual behavior.

  • Gamify to suit your audience

Making tasks more intuitive, and fun is a way of holding attention levels high – TNS sees gamification as an area of opportunity.This can be very simple gamification – removing a 1 – 5 touchpad option to a slider for example.

Challenges for Mobile Transition: Audience Biases

TNS currently see limitations in the size and scope of available mobile-enabled panels.

Where mobile panels are available, they see certain audience skews:

- more tech-savvy

- less older respondents

- less younger males

No doubt this is a moving target, with many panel providers working actively to increase the number of their participants who are opted-in for mobile research.

Given the current paucity of mobile-enabled panels, TNS currently adopts a partnering approach, linking up with companies outside of the traditional MR space that can deliver potentially interested MR audiences with the right approach and incentives.  These partnerships are becoming increasingly fruitful in Emerging Markets like India.

Mobile  & Passive Monitoring?

Kantar has been collecting “clickstream” data – respondents’ complete use of their mobile – for over three years now in North America.  They record telephone calls (only duration, not the content of the calls) pics taken, sites visited, ads exposed to -  recorded passively by an App they download. Cell tower GPS information reveals where respondents are.

A pilot is also testing an audio recognition app that can listen to and record sounds in the immediate vicinity – similar to the music-recognition app Shazam.

Such tracking allows a comparison between claimed and actual behavior – and is powerful in detecting potential contradictions – useful for insight generation.

As a method, passive mobile monitoring is something TNS is treating as an area of huge potential. There are still challenges in persuading respondents to join these panels over privacy concerns, even though no personally identifiable information is stored.

Whether sufficient consumers will consent to total tracking in future, thereby permitting some degree of representativeness, is an open question.

Unless barriers are overcome, passive monitoring may remain of niche relevance for research.

Summary/ Outlook

  • With its ongoing global mobile research program amongst tens of thousands of mobile phone uses, TNS is well positioned be a global leader in understanding the habits, preferences and  movements of the mobile consumer.
  • Using mobile as a vehicle to help convince the industry (clients and Agencies) to shift to shorter surveys with higher predictive validity has multiple benefits. It also has the potential to revolutionize the way surveys are conducted, possibly signaling the demise of the long, tedious 25 + minute tracker.
  • Mobile should improve touch-point and customer journey analyses. Improved efficiencies in media and marketing mix planning should result from mobile’s ability to register more touch-points, better understand how touch-points interrelate and then lead to an event such as purchase, and the context in which someone felt good or bad about a brand or a marketing message.
Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

The ARF Captures Our Marketing And Research Industry Journey

This is the first part of a two part blog series on learnings from the ARF Rethink 2014 conference. Part two will be on “Big data, big research possibilities”.


By Joel Rubinson

The ARF Rethink 2014 conference did a great job of capturing our marketing and research journey.  The keynote address on day one was actually an interview of one of the real 60s “mad men”: Keith Reinhard, Chairman Emeritus of DDB. Soledad O’Brien conducted the interview to artfully extract his view…of course we need research, but it was clear that his focus was on the creative spark that led to “…hear(ing) the bat hit the ball and you know it’s a home run.” So we started the conference with paying homage to ‘the great TV ad’, in a world of single-screen behaviors and little acknowledgement of the contribution of research.  It was a world where brand building was centered on the TV commercial, catching lightening in a bottle, and where there was no guarantee that success on one assignment would be reproduced on the next.

That was the past. But then, the ARF brought us into the present and near future.

At the conference we heard from Facebook how dominate mobile use is becoming to overall Facebook access and the need to link behaviors across screens for the same individuals via persistent log-in. They shared a factoid that 40% of people begin an activity on one screen and then complete it on another.

While we are in the digital age, at the same time, we saw evidence that it is also the golden age of television and in fact, digital might be TV’s best friend. Dave Poltrack of CBS shared three startling facts:

  1. TV viewing hours are NOT declining and linear TV is still, by far, the dominant behavior
  2. TV program audiences are NOT declining when you consider the full reach of a program across TV, social, digital, and mobile interactions
  3. There is a clear and conclusive correlation between program engagement and the effectiveness of advertising run on highly engaging programs.

The takeaways: advertisers can neither ignore the power of TV, nor take it in isolation of simultaneous second-screen behaviors and creatives need to realize how much tougher the challenge has become…to create ideas that work across screens to amplify and reinforce one another.

As media behaviors evolve in our digital, social, and mobile age, the marketing questions change that research must address while also arming research with tools it never had before.

The ARF showcased new possibilities for measurement from best in class work that leveraged big data.

We heard about the following data streams being leveraged for breakthrough insights:

  • Social media listening to naturally occurring conversations and vocabularies (Occulus 360)
  • The use of web-based micro-surveys to build a previously unheard of library of nearly 30,000 questions that offer the richest playground imaginable for using data science to mine for unexpected insights (CivicScience)
  • A new science was described and validated, “expectation science” that is the needed companion to “measurement science”. (CivicScience)
  • Bringing consumer segmentation to life not via focus groups but by creating prototypical digital behavior patterns (Brainjuicer’s Digividuals applied to Allstate segmentation)
  • The merging of massive databases of media behaviors, attitudes, purchase behaviors, media spend, etc. using anonymized matching and data fusion methods  (Nielsen, IRI, Comcast)
  • The power of asking questions to those whose digital behaviors are tracked to create a single source way of understanding path to purchase digital behaviors and motivations (Luth on behalf of Ford)
  • The challenges and opportunities of conducting research via smartphones.  AOL showed powerful evidence that smartphone research participation rates are lower, satisfaction with survey taking is lower, but data quality can be higher.
  • The continued need for the long-form survey against certain business questions…CBS conducting a 40 minute survey among 7,000 respondents, as part of a research program that included merging massive data sets together to understand the landscape of video viewing motivations and how program engagement affects advertising effectiveness

Ask yourself: do you have a data strategy to generate insights and measurement that leverages every one of these arrows in the quiver or are you still primarily in a traditional research mode? Are you working as hard to understand the sea changes in media consumption as you are to understand consumption of your brand?

In the next blog in this series, I will describe in more detail some of the emerging big data and data science-based solutions that were described at the ARF Rethink 2014 conference.


Announcing The Finalists Of The 5th Wave Of The Insight Innovation Competition!

After several weeks of intense competition in the 5th wave of the Insights Innovation Competition, the results are in and 6 finalists have decisively risen to the top of the list based on votes by you, the global market research community.


After several weeks of intense competition with 7 firms competing in the 5th wave of the Insight Innovation Competition, 24,167 views and 3,100 votes the results are in and 6 finalists have decisively risen to the top of the list based on votes by you, the global market research community.

Interestingly, although the number of submission was down from previous waves (an artifact of this judging round happening in Chile, I believe) the numbers of views and votes are up, so these seven fought hard.

Please join me in congratulating all of the competitors and the finalists. Here they are:


eCForce Adriana Rocha 14503 1656
SocialDecode Eduardo De Leon 5940 482
Survmetrics, interactive surveys. Ramon Escobar 2112 471
Rolling Labs Mike Courtney 846 416
Sustainable Research Fiona Blades 642 64
Vysical Labs: Immersive Video Games for Product & Retail Testing Rolfe Swinton 124 11


The submissions broke down into a few key categories:

Social, Local & Mobile: 1 submission (eCForce)

Big Data Analytics & Visualization Tools: 2 submissions (SocialDecode & Rolling Labs)

Gamification: 1 submission (Vysical Labs)

Adaptive Surveys & Agile Data Collection: 2 submissions (Survmetrics & Television Surveys)

Other: 1 submission (Sustainable Research)

So a pretty wide swath of “buzz terms” were covered with a great variety of approaches and technologies represented. What is most interesting (to me at least) is how this mirrors broader trends both within and outside of the insights sector: these folks are obviously paying attention to what’s happening and are building businesses that are positioned to tap into those trends.

What happens next:

For the finalists, they are going to present on stage in front of our panel of judges and the audience at IIeX in Santiago, Chile in April. This is a no-lose proposition for them: past participating companies have seen their businesses accelerate due to their involvements in the Insight Innovation Competition, resulting in funding, partnerships, new clients, and global brand exposure. For the single winner they will get:

  • Exposure to a large international audience of potential prospects, funding partners and investors, including the Ricoh Innovation Accelerator, Lowe’s Innovation Lab, and independent venture capitalists and angel investors
  • A free consultation provided by Gen2 Advisors to help the winner develop a growth and expansion strategy.
  • A “Hot Desk 60”membership for one year at the Center for Social Innovation’s newest facility in the iconic Starrett-Lehigh building in Manhattan, which includes 60 hours of coworking space and 3 hours of meeting room access per month
  • An invitation to present at the next Insight Innovation eXchange
  • An interview to be posted on the GreenBook Blog, viewed by 36,000+ industry professionals per month
  • An opportunity to work with successful senior leaders within the market research space

On April 9th one of the above companies will join Decooda, Zappistore, Raw Data, RIWI and SocialGlimpz as winners and can look forward to rapid acceleration of their business.

competition winners

For the runner-ups (and the finalists who don’t win the next round too), the news is still good. All participating companies will be vetted for inclusion in the Ricoh Innovation Accelerator and Lowe’s Innovation Lab programs. Selected participants will gain guaranteed organic funding through pilot programs with program partner companies, as well as access to acceleration resources for marketing, strategy, finance, and business development.

The next round of the Competition will launch in April aligned with IIeX North America in June.   Stay tuned for more details on that soon.



Congratulations to everyone who competed and good luck on the next steps for each of you!


It’s New And It’s Shiny – - So What?

The rallying cry that spurs interest in new and shiny things is “Innovate or Die”. Here’s the problem – there’s no validation of this statement.



By Steve Needel, PhD

C-Suite folk and their minions, the marketing team, have always had a fascination with new and shiny things. This is especially true, it seems, when they look at marketing research technologies. A couple of years ago, I gave a talk at the MRMW/ US conference on the misuses and misrepresentations of eye-tracking, neuromarketing, and facial recognition. At MRIA and ESOMAR last year, I questioned whether Big Data is as big an issue as we seem to be making it. My goal in each of these talks was not to try and kill off these new tools. What I wanted to do was provide some guidelines for how we as an industry should be evaluating these new ideas. As innovation is the theme of a recent conference in Amsterdam and an upcoming one in Atlanta, it might be time to further explore these guidelines; especially because that’s what Lenny pays me to do (okay – he really doesn’t pay me much – an occasional promise of lunch).

The rallying cry that spurs this interest in new and shiny things is “Innovate or Die”. Here’s the problem – there’s no validation of this statement. While self-styled authorities such as Tom Peters have been saying “Innovate or Die” for years, Getz and Robinson (2003) actually studied the question of whether innovation leads to a long and happy business life. Their conclusion is simple – it’s not innovation that makes a company healthy, it’s the system they have for improving products. They point to Xerox, which may have been the most innovative company in the world in the 60s and 70s, and Alcatel, who was in the top three for communications technology in the 90s, as two of the great disasters of failed innovations. Their problem, according to the authors, was a set of innovations that nobody wanted.

Our tendency in marketing research is to take any new technology, hype its value beyond its original intent, then backpedal when we find out the emperor didn’t have as large a wardrobe as we thought. Neuromarketing has gone through this pattern in the last two years, with the extravagant claims of the charlatans largely disavowed by most in the business, even by those selling that stuff. We went through this last year with Big Data, when much of what was being promised turned out to be Big Hot Air. In both cases, we’re getting down to the real work of determining what we can learn from neuroscience and the analysis of large, semi-structured data sets and the perimeters around those tools. The problem remains though; marketers and executives seem to believe these are critical factors in their business and when we can’t deliver, it’s the research industry that takes the hit.

You can’t blame the executives for this [much]. They shouldn’t be expected to understand the intricacies of research in the same way we researchers often don’t understand the rules of accounting, the science behind logistics, or the legalities of human resource management.  Their job is to make decisions for improving their business; our job is to give them information that will inform those decisions. That information may be situational, as in, “here is where the marketplace is and where we are in that marketplace”; it may be proactive, as in, “we’ve identified an opportunity”; or it may be reactive, as in, “we’ve tested this marketing idea and here’s what we can expect consumers to do”.

So where’s the disconnect? Mostly it comes from the perception that we don’t do these things as well as they need or in a time frame that fits their perceived needs. This opens the door for anyone who promises to fill those needs, whether they can or not.  And this is where innovation often goes wrong and gives marketing research a bad name. It may be trite, but innovation needs to produce methodologies that are faster, better, or cheaper. Faster and cheaper need to be at least as good as what was there before.  Better needs to be demonstrably better, not just theoretically better. This is where I think we have missed the boat – we all too often sell what’s new and shiny rather than selling a better tool.

What makes a better tool? That’s easy – one that more accurately predicts what shoppers/consumers will do. The promise that many (not all) suppliers in the eye-tracking, neuromarketing, facial recognition, mobile research, and Big Data space make is that they have the technology to tell you what people will really do. The problem is mostly it’s not true. There is no data that says more attention or longer attention to a product on the shelf improves the probability of purchasing. There is little data to suggest that neurological or physiological patterns are better predictors of purchasing than well constructed survey or experimental techniques. The basic tenets of facial recognition are now coming under attack – they may not be as universal as we once thought and again, nothing has been published to say it is a better tool.

Just being new, just being different, just being sexy with lots of sizzle sells; it certainly has in the past and probably will in the future. But if MR is really swirling down the drain, then we may be cutting off our noses to spite our faces. Selling cotton-candy techniques is only hurting us. Considering them to be innovative just because they are new and shiny misleads those who we most need to trust us.  We can do a better job and we must do a better job of innovating.


Mobile Market Research – Improved Touchpoint Analysis

A Look at Mobile Market Research & Touchpoint Analysis – Conversation with Fiona Blades, CEO of MESH.



Editor’s Note: Edward Appleton is doing a series of posts focused on the client-side view of mobile research, with an emphasis on use cases and best practices learned so far. This is the third  post in that series of posts that we’ll be publishing over the rest of the month. Part 1 & 2 can be found here.


By Edward Appleton

MESH is a relatively new but powerful player in the Research space. Founded in 2006 by Fiona Blades, the company pioneered the use of mobile in touch-point analysis.

Its central tool – Realtime Experience Tracking (RET) – was described by Harvard Business Review as “A New Tool radically improves Marketing Research” (09/ 2012). With its head offices originally in London, MESH has now expanded internationally, with offices in Singapore, New York and Sao Paolo. The company works for major multinationals including PepsiCo, Diageo and Unilever, LG Electronics and BSkyB – pretty impressive for a company that has existed only 7 years.

Fiona Blades, founder and Chief Experience Officer, shared her thoughts with me on the role of mobile in market research – or experience monitoring, in MESH parlance.

Why Mobile?

The origins of MESH and mobile tracking go back to the early 2000s. In her capacity as an advertising campaign planner for UK ad. agency Claydon Heeley she consistently noticed how traditional advertising tracking monitoring wasn’t picking up on the full range of brand touch-points.

“I knew from friends, and personal observation, that it wasn’t just TV advertising that was influencing a purchase decision of, say, a car. Seeing the neighbors’ car, going online, reading reviews…. there were many influences that we simply weren’t picking up” is how Fiona described it.

The overall picture of communication influence wasn’t complete – touch-points were getting lost. This was mainly due to two issues:

  • Memory gaps: respondents have flawed recall
  • Methodology limitations: direct questioning methods are limited in their ability to assess the true impact of a whole range of touch-points. In direct questioning, we overstate some media (e.g. TV) and understate others (e.g. outdoor, social media)

The critical commercial insight was that companies were missing potential opportunities.

This prompted Fiona to explore options to plug the touch-point gaps.

Piloting Mobile: Superior Accuracy and Speed of Insight Delivery

In 2001, Fiona conducted a one-off experiment in soft drinks communications research to validate what was initially a hunch. The use of mobile was central to the new approach: respondents were briefed to behave like researchers – taking their mobiles with them at all time, recording all their experiences on the mobile in a minimally intrusive manner.

The experiment suggested there was an idea there: when recording experiences on their mobiles, a higher number of touch-points with a greater diversity was registered than traditional touch-point methodology. The role of the “supporting media” – whether it be POS, outdoor, ambient to name a few – was up-weighted, the overall media picture more accurate.

And critically for decision making: the reporting was real-time, results were available in a matter of days, hours even, far quicker than traditional tracking reporting.

This initial success encouraged her to work on a simple tool designed to utilize mobile to capture experiences – Real-Time Experience Tracking (RET – see below ) and officially start Mesh Planning in 2006.

The Real Time Experience Tracking tool (RET)

Central to Mesh Planning is their “Real time Experience Tracking” tool. The technique is eminently simple and comprises of three distinct phases:

- Phase 1: Brand understanding: typically done on the laptop/desktop, this elicits data on brand awareness, knowledge, usage, perception.

- Phase 2: Mobile Capture – Diary: participants are asked to record every encounter they have with a category, wherever they are:

  • Which brand?
  • Occasion?
  • How does it make them feel?
  • Persuasiveness: how likely to choose next time?

The options are limited, but include an “other”, and are pre-coded with a letter or number, so each respondent only gives a set of four digits. This is designed for speed and ease of response.

During this phase, respondents are also asked to maintain an online diary, typically on their laptop, where they are asked to describe experiences in more detail, upload photos. Probing questions specific to a client can be included here. Doing this on a smartphone would yield far less rich data, hence the use of traditional online.

Phase 3 – Data capture, again relating to brand knowledge/usage – and mirroring the questions posed in Phase 1. This allows a measure of pre-post shift.

Respondents are treated in a different way in this tool: they are briefed to become a researcher. This heightened involvement helps ensure richer responses, and is designed to pick up far more accurately on all and every touch-point.

Mobile Touch-point Tracking: Advantages

The features and core benefits of the  MESH touch-point tool are as follows:

  • Immediacy of response: mobile captures the reactions in-the-moment, including the way it makes respondents feel. Little is lost due to memory gaps.
  • Speed:  mobile data capture and analysis is far faster than traditional tools. Feedback and top-line is available in days rather than weeks. This has major economic implications.
  • Full coverage: mobile is always with one, always on -  it captures all experiences, not just those that are most salient and mentally available.
  • Enhancement of communications effectiveness: RET measures the relative influence of touch-points on the likelihood to choose a service or product in future. This allows under-performing touch-points to be quickly identified, and an optimized mix introduced.

As media continues to fragment, this ability to fully cover experiential touch-points – as opposed to media-driven ones such as TV – is a key advantage of mobile.

Quant or Qual? Duration of Study?

Mesh studies are very largely quantitative with photos and comments included to give a qualitative perspective, a first-take on answering “why?”

A project  typically lasts six weeks, with each respondent engaging for a week.

What Typical Types of Research Is Mobile Relevant For?

There are two main Research areas that MESH’s mobile experience tracking is used for:

  1. Marketing and Campaign evaluation
  2. Shopper Research – mapping the path to purchase

i) Marketing and Campaign Evaluation:

Thanks to the speed of data-capture, and the improved accuracy of touch-point pick-up, the tool gives marketing decision makers a much greater ability to optimize their campaigns faster.

Touch-points that are under performing can be ditched, monies re-directed to areas that resonate strongly.

This results is clear and measurable improvements in ROI

Importantly: budget re-allocation is possible (at least theoretically) during a campaign. This saves immense amounts of money being wasted on the wrong media.

Case Study Gatorade Mexico – Fine-Tuning a Re-Positioning Exercise

Gatorade in Mexico had decided to re-position from a general sports drink to a specialist sports nutrition product.

Whilst the company possessed much research data on the re-positioning message, their decision to invest in experiential  touch-points such as parks and gyms was less well grounded in empirical evidence.

They tasked MESH with executing a mobile-based study, asking the key question: was this channel-based investment working?

Mesh recruited 400 respondents, asked them to text about their experiences and interaction with Gatorade and its competitors over a one week period (the project duration was 6 weeks in total), and complete questions about brand image both pre and post the mobile diary phase.

The insights generated were as follows:

  • confirmation that a broad range of Gatorade drinkers could identify with the new sports-specific messaging
  • identification of strongly performing “Gatorade” experience touch-points
  • shift of media expenditure from traditional media to those touch-points, improving the media mix effectiveness
  • demonstrated how one product in the range was insufficiently understood; shifted communication focus to this drink
  • highlighted opportunities to improve the Gatorade website, engage in store, utilize outdoor: all points close to the point of need

The marketing team took action immediately on these insights. The launch proved hugely successful, and became the blue-print for the Gatorade roll-out in Brazil.

ii) Shopper Analysis: Tracking the Path to Purchase/Choice

The RET tool allows brand owners to see exactly which touch-points are used more, which less over a period of time leading up to a purchase event.

Using mobile research to helps create a distinctly granular path-to-purchase that previously was not available in that level of detail. Brand owners can optimize their Customer Decision Journey model, and create distinct journey paths for different segments of shoppers.

The interplay between online and offline can be well understood. Behaviorism data (through cookie tracking) can be overlaid onto attitudinal data.

The following case study shows how mobile tracking can be used not just in brand advertising, but also for political campaigns.

Case Study: Tracking Floating Voters’ Political Opinions before the UK 2010 National Elections

MESH joined up with Cranfield School of Management  to pioneer a new way of monitoring the shifts in voting intent amongst “floating voters” in the immediate run up to the 2010 UK elections.

The RET system was adopted. 900 British floating voters recorded their perceptions (over 18.000 experiences registered) of the main political parties Conservatives, Labour, Liberal Democrats (Lib Dem) over the period of final weeks in the run up to the British election (6 May 2010).

The key objectives were to identify:

  • which messages from which party had an impact on switching voters
  • gain/loss analysis of the major parties per major communication effort
  • which touch-points were effective with what messages for what party

Some key insights were as follows:

  • Social media was underutilized: only 3% cited SM as a source of influence. This is in stark contrast to the 2008 and 2012 US elections, which proved the power of social media in political campaigning.
  • Floating Conservative voters who saw a Lib. Dem party political broadcast were five times more likely to switch to be Lib Dem voters than those who didn’t see it.
  • Floating Lib. Dem voters who saw a Conservative broadcast were far less likely to vote Conservative. Respondents indicated they disliked the negative campaigning style.
  • Conservative posters had a strong positive influence on Lib. Dem floating voters.
  • Live televised debates and the reporting on them the day after in national newspapers had a strong impact on floating voters, favoring the Conservatives and the Lib. Dems.

What is striking is the granularity of the insights delivered, and the clear attribution to a particular touch-point and message, allowing better targeting of marketing expenditure.


MESH see mobile as continuing to grow in importance as a mode of data collection. What was originally considered a gimmick is now accepted as mainstream – with the publication in HBR a critical milestone in gaining credibility.

  • Noise around mobile isn’t hype – the continuing growth of smartphones and tablets effectively changes the way we experience the world. Research needs to mimic real-life experience.
  • In emerging and fast developing markets – many countries in South America and Africa – mobile is actually the only form of data collection possible. This automatically makes mobile a must in these growth markets.
  • The added value comes, in MESH’ view, less in how the data is collected, more in the way mobile data is analyzed. They have developed a suite of advanced analytic tools allowing a sophisticated mining and cutting of the data, as well as an overlay of other data sources such as behavioral.
  • Predictive modelling using mobile data is a powerful tool, allowing clients to experiment with different scenarios of media funding allocation, showing the likely impact on preference.
  • Understanding more closely how experiences interact – when one experience leads to another – identifying patterns, forming clusters, is another area where MESH anticipate growth.

Curious, as ever, as to others’ views on mobile market research.


Six Lessons From The MRS IMPACT 2014 Conference

Tom Ewing gives his take on the MRS Impact 2014 event and lists his key takeaways.



Editor’s Note: Tom Ewing is one of my favorite bloggers; his erudite and no BS persona always ensures a provocative and entertaining read.  Since I couldn’t attend this week’s MRS conference myself, there was no one better to be our “eyes & ears” at the conference and ensure that we got the best possible take on the event. It sounds like it was a great conference that really delivered on the title of “Impact”. You can find more coverage here.

For the record, GreenBook and MRS have a very friendly relationship and are looking for even more ways to collaborate in the future. While I appreciate Tom’s comparison of MRS and IIeX at the end of his post, increasingly we’re looking at working with trade associations to help them leverage the types of things we uncover at IIeX and incorporate them into their own unique value proposition for their events.    No one event or even media platform can be all things to all people, and although we’re pleased folks find value in what we are doing here and that it is influencing others, our goal is to help the industry as a whole grow and prosper, including our trade organization partners.


By Tom Ewing

In an ever-more crowded conference schedule, it’s more important than ever to stand out. Over the years, the MRS’ annual conference has found various ways of doing that. In the less globalized industry of the 1990s, simply being the UK’s biggest research event was difference enough, and the conference got a reputation for being a little bit staid, with most of the real action taking place in the hotel bar.

That all changed in the mid-00s, when new organizing blood, and pressure from a body calling itself the Research Liberation Front, pushed the conference towards a more radical agenda. It became known for big multi-disciplinary ideas, highly creative session formats, and an eclectic choice of keynote speakers. Robotics would sit side by side with semiotics, book clubs fought for audience with pecha kucha, and you’d finish off with a keynote from the likes of comedian Armando Iannucci or socialist icon Tony Benn.

The only question – and it was a legitimate question – was “what does all this have to do with research?” For every attendee who let the MRS spark their imagination, another felt a lack of practical applications. So these days, under CEO Jane Frost, the MRS takes a third way – the slogan is now “evidence matters”, and the mind-expanding keynotes and far-out workshops are now balanced with an emphasis on really solid case studies.

So at the 2014 conference this week – called Impact 2014 – we had Cadbury’s talking about the ROI of Facebook content marketing, Waitrose extolling customer closeness sessions, eBay singing the praises of its segmentation, Greenpeace using psychology to make its Facebook page more inviting, and several smaller UK businesses – like an independent whiskey distillery – showing how research had paid off for them. As a set of examples of the real value of research, it’s hard to beat.

But, weirdly, the old question still lingers – case study heavy sessions are obviously about research, but they aren’t always relevant to the day-to-day work we do. You’re looking for the insight that will move you from celebration to transformation – something to inspire change in the way you do research.

So with that in mind, here were the best tips and ideas I personally took away from the conference.

  1. ALWAYS BE TESTING: A/B testing was a constant presence at Impact 2014 – the idea that, like the UK cycling team discussed by keynote speaker Tim Harford, constant marginal improvements can deliver decisive gains. As he was quick to point out, that’s far from the only way to innovate, but the idea cropped up all through Impact 2014. You saw it with Cadbury’s, who used real-time Facebook audience engagement as a substitute for pre-testing: launch a whole lot of content and then back anything that starts to do well. You saw it in the work showcased by The Behavioral Architects on Priming – small, background sensory differences can have a very large impact on how an experience is perceived and enjoyed. If you’re not getting creative about marginal improvement, you’re leaving opportunity behind.
  2. STRUCTURAL BOLDNESS: Tim Harford contrasted his talk of marginal improvement with the story of genetics pioneer Dr. Mario Capecchi, who overcome astonishing hardship in his childhood to make brilliant scientific discoveries. It was a classic tale of visionary boldness set against incremental progress, and the audience were soon on the side of the bold individual. But, as Harford pointed out, the binary is the problem – we love those stories because they seem to justify a system where we make bold individuals shoulder an enormous financial and personal risk. If we created structures which invested more in bold experiments, they’d seem less exciting, but we’d probably get more done.
  3. BEWARE STORIES: Two of the conference’s other keynote speakers also had plenty to say about stories. In an entertaining talk about just how much higher math there is in The Simpsons, Simon Singh suggested that simplification by itself isn’t always an ideal goal. You can fit all sorts of complexity in there as long it’s in service to the story (and in the Simpsons’ case, the gags). But novelist Will Self took a more skeptical view of stories. Narrative is dangerous, he pointed out – it’s a device for cutting down options, and making outcomes seem inevitable. That’s what makes it so valuable for presenters, but a false friend to consumers – and perhaps to researchers, who can be seduced by the stories they’re trying to tell. Self’s points were reminiscent of Daniel Kahneman’s pessimistic assessments of story in Thinking, Fast And Slow – ironically, since Self criticized behavioral economics for its emphasis on our difficulty in making “informed choices”. Do marketers want people to make decisions against their will, he asked? To which the researcher might say, it’s not about how we or Will Self would like people to decide things – it’s about how they do decide them. But Self’s wider point is very true – marketers tend to bandy the word story around with no real understanding of its implications. If you’re offering choices and a customer doesn’t like one, you can always offer them another. But if you’re telling a story and someone doesn’t like it, you’re finished.
  4. TRUST YOUR STAKEHOLDERS: One way of improving decisions, according to Kahneman, is to get a more neutral opinion on them. I was reminded of that during a fascinating panel about the rise of activism in 00s culture, spurred partly by the speed of ideas on social media but also by more powerful NGOs. It’s long been said that the most talented minds go into business, not politics. But now, said Robert Blood of Sigwatch, the most talented brains also go into NGOs not politics. Companies are often well behind the modern NGO in terms of media reach and understanding, and Sigwatch helps them keep track of issues before they hit the media (and become boardroom issues). What’s interesting is that a direct effect on share price or sales is incredibly rare, but businesses change behavior anyway because of the psychological leverage activists have and their influence on public opinion – few people, no matter how hard-nosed they are, want to be seen as the bad guy. The problem, said the panel, is that companies then look to NGOs for advice – which isn’t an NGO’s job. An NGO’s purpose is often to criticize corporate behavior, not to hand out pats on the head. An NGO is not a stakeholder. So instead of looking at things through the NGO lens – which means either defiance or capitulation – companies should make sure they have good communications channels to their actual stakeholders: owners, employees, suppliers and customers. That’s where good decision advice will come from. Which, of course, is where research comes in.
  5. GO “NODE TO PEER”: Sometimes stakeholders aren’t easy to find. In a case study that was nominated for the Best Presentation award, by Southwark Council (a London local governmental body) and ESRO Research, we learned how to use “node-to-peer” research to look for them. Like London as a whole, the borough of Southwark is home to a huge range of cultures and immigrant communities, and some in particular are poorly known to the council and government services. By recruiting “cultural brokers” from community workers to Imams as ‘nodes’, ESRO was able to supplement its ethnography and workshops with more quantitative data, ending up with more than 900 very hard-to-reach respondents.
    The ESRO method enjoyed some of the most positive Tweeting of the conference, and excited attendees well beyond its social research origins. With similarities to techniques used in developing market research and youth research, it was an excellent study in how to turn the theory of ‘influentials’ into something more material than just a field in a database. In the future, the data collection end of custom research is likely to become more reliant on difficult to identify populations as a value add – “node-to-peer” is one way to do this.
  6. WIDEN OUR INPUTS: But perhaps node-to-peer is something the industry could be doing more of itself? One of the most interesting panels of the conference addressed diversity, and whether the research industry could be doing more to increase the range of people working in it. An audience member’s comment that the UK industry is “horribly white and middle class” caught most Twitter attention, but this was a slight misrepresentation of a panel that was more about pointing out good examples and suggesting solutions than it was about castigating anybody. Sam Phillips of Omnicom, Ettie Etela of Nielsen and entrepreneur Belinda Parmar praised brands like M&S for better representation in advertising, discussed the importance of role models, and suggested apprenticeships as a way of breaking research’s dependency on university graduates. Like most good panels it was over too soon, but this was an important panel on a vital subject. Diversity as a research asset goes beyond political opinion: the more work I do in behavioral science, the more I know how very contingent our interpretation of data is on our subjective assumptions, and how difficult it is to step beyond those. Diversity helps solve that problem: the more our industry looks like the people it studies, the better our work will be.

So those were my take-aways. A quick look over them should tell you that the MRS remains a conference with distinct and individual strengths. It’s not afraid to be intellectual, and it’s not afraid to challenge or to entertain. It also has a strong tradition of panels that deal with political issues and with the material lives of ordinary people – a legacy of the UK’s important social research sector. And it has a good grasp of psychology too – this was the first conference I’ve been to where a session on behavioral economics could assume everyone already knew what System 1 and 2 were!

But it’s also lacking in some areas – particularly in the kind of things Greenbook covers. The MRS has never been great on mobile research, but having come fresh from IIEX in Amsterdam it was startling how little discussion there was of technology in general, and the new entrants to the industry that it’s creating. Individual companies and developments can be overhyped – there’s only so many times a game can change – but industry shifts are real. Even though most presentations were very strong, there was a rather cosy feel to Impact 2014, a sense that one can ignore the winds of change by settling down with a well made case study and a thoughtful cup of tea. If the MRS could combine what it already does so well – its solid examples, its intellectual curiosity, and its grounding in consumer reality – with some tech adrenalin, it might well be unmissable.

Thanks to Joseph Clift from WARC for his notes which let me write up the ESRO presentation.