1. Clear Seas Research
  2. SIS International Research
  3. Research Live
  4. RN_In_App_Effectiveness_GBook_480_60

Facial Imaging: The “Big-Data” Solution for Emotion Research?

We are entering an era where, thanks to technology like facial imaging, “soft-data” on emotions – traditionally the province of qualitative studies or smaller scale specialized surveys - will become “big-data” that provides very hard results.

facial scanning

By Alastair Gordon

We are entering an era where, thanks to technology like facial imaging, “soft-data” on emotions – traditionally the province of qualitative studies or smaller scale specialized surveys -  will become “big-data” that provides very hard results.

At first glance facial imaging (or “facial coding”) seems like just another variant of Neuroscience testing, but in fact it has some very different features. In earlier posts we’ve written extensively on the results obtained from this technology (e.g. see “Soft-Drinks, Soft-Sell“), but in this post I want to get across the point that the really big news is not so much how well facial imaging measures emotion, but how many people and how much emotion can be measured.  This makes it fundamentally different from hardware dependent methodologies like EEG or conventional survey based methods. Two thought experiments for market researchers might illustrate:

  1. Imagine you could take 100,000 people through a series of tasks of your choosing while simultaneously collecting accurate emotional response data on their responses. Further, add to that dream, that minimal direct incentives were necessary to collect that information? What kind of research solution could you make out of that ability? ……..
  2. Now put yourself in the place of a medium sized MR agency trying to compete with the big players in advertising evaluation. What if you could measure reaction to every ad in a category (your client and their rivals) across three countries simultaneously and get results back to the client in days, not weeks. Add to that idea, the supposition that the fieldwork cost of all that is about the same as others take to collect information on 1-2 ads. What would that let you discover about how advertising really works in your client’s category? Would it make you more competitive?
cinemax-shot

Facial Imaging Embedded & Automatic: nViso API in Cinemax site – 1 million visitors and counting

Neither of these ideas is a fantasy with the use of facial imaging technology.  nViso’s 3D Facial Imaging solution has been integrated into a project with Cinemax (http://www.google.com/think/campaigns/cinemax-hunteds-byzantium-tests.html) in which viewers effectively play a “spy game” and have their emotional reactions recorded at various stages for entertainment purposes. In the viral campaign they analyzed over 1 million people in the first 10 weeks of the campaign. It doesn’t take much imagination to see how, with a bit of “gamification” expertise, that this kind of scalability could be the basis for a powerful choice modelling/prediction solution.

Comparing  Sadness Response- 3 Videos In China. The AsiaEmotion study looked at 75 ads across five countries

Comparing Sadness Response- 3 Videos In China. The AsiaEmotion study looked at 75 ads across five countries

Similarly at Gordon & McCallum we recently worked with Cimigo (a leading Asian branding and MR agency) and nViso to conduct a project across five Asian markets in which we evaluated 75 TV ads in a single study. Because we could easily expose each respondent to multiple ads, the fieldwork effort in collecting this information was a fraction of that required for 75 conventional monadic tests.  Yet consider for a moment the scale of data resulting from such a study – second-by-second measures of seven specific emotions recorded passively across multiple ads and categories. It should be obvious that the potential for sophisticated analysis and modelling is vastly greater than that obtained from a standard set of survey responses.

The point to note about both these examples is that facial imaging is not a “research product” as we are used to thinking about them – it’s a technology platform that can serve a variety of research purposes: a media company can integrate it into a game on a website, or a MR agency can pull it into a concept-test study. The pan-Asian study noted above was conducted via CLT (Mall) tests, while many other studies have been done online, so geography is not an issue.

We also like the idea that the technology is “marketing theory agnostic” – essentially it produces a bunch of numbers reflecting the level of emotional response to a stimuli – these can be cross-analyzed by any questions you like, or integrated into all sorts of research models. While it does require stimuli, this doesn’t have to be an advertisement: concepts, words, packs and videos can all be used.  Possibly most importantly, this is an ideal format for clever experimentation – innovative researchers will soon, we believe, be creating their own videos as stimuli – like collages in today’s Focus Groups such videos will mix dozens of branding, positioning and imagery elements; but unlike a focus group, facial imaging technology will provide far more statistically robust conclusions on which combinations of marketing inputs are most effective.

What all this means is that brand managers and researchers need to think outside the “neuro-testing” box when it comes to considering the uses of facial imaging technologies. While we think EEG is very powerful for many purposes, it isn’t going to collect data online, or be integrated into a major tracking study, and it isn’t likely to be cost-effective for work in Lagos, Lucknow (or possibly even Little Rock!). The big point about facial imaging then, is that it is taking accurate emotion analysis into settings it could not go before, with sample sizes that have not previously been feasible. This is truly a transformation of how we think about collecting “soft-data”.

It’s also accessible in the sense that it’s not too expensive and is easy to implement — so any brand manager or MR or advertising agency anywhere in the world can give it a go! Those that do, and use it imaginatively, are likely to find that it provides a real competitive advantage.

For an introduction to facial imaging and it’s possible applications, we’ve created a two part presentation:  Reading Faces: The Art & Science of Facial Imaging in Market Research” on Slideshare:

http://www.slideshare.net/AlastairGordon/how-to-look-at-facial-imaging-results-part-1-basics

http://www.slideshare.net/AlastairGordon/the-art-science-of-facial-imaging-part-2-advanced

For more on the AsiaEmotion study, including several case studies and an overview of our conclusions, see AsiaEmotion.com)

Or feel free to contact us If you have queries about the technology at info@gordon-mccallum.com.

Share
You can leave a response, or trackback from your own site.

9 Responses to “Facial Imaging: The “Big-Data” Solution for Emotion Research?”

  1. Chris Robinson says:

    September 5th, 2013 at 11:11 am

    Alistair, I think there are a lot of readers out there who will be re-reading and re-reading again your article to find out what makes you so excited, just what do you do with all this data? To but it tritely having lots of participants and lots of data sounds great but what decisions can you make from it? Surely there is little objective insight if you know an emotion is linked to a scene in an ad. Does that mean it is a negative or a positive? How does anyone interpret that, or is it more qualitative in that it might mean something but its up to the interpreter.

    I think the technology is great but it is in serious need of validation against real marketing measures and not how some ad or MR agency feels about the “insights”.

    Actually that raises a couple of questions. Why for example would a client actually test a finished commercial. Surely the money has been spent and the real results are in the market. And just what is the ad agency position on this? I would guess they see this as bit more voodoo stuff from market research.

    Please enlighten us on the practical applications of this .beyond being a nice add-on?

  2. Steve Genco says:

    September 5th, 2013 at 1:42 pm

    Alastair posted this article on the Marketing Science discussion group on LinkedIn, and I posted a comment there. Re-posting here to reach the GreenBook community. I agree with Chris Robinson that there are important practical questions as yet unanswered about this technology, but I feel more positive about its potential and anticipate a significant impact on the industry due to its scalability and low price point. This looks like a classic disruptive technology in the Clayton Christiansen sense.

    My post on LinkedIn: Alastair, thanks for posting this very interesting article and associated presentations and case studies. …

    In addition to enjoying the studies themselves, which are quite clever (love the Cinemax promo with its gamification elements), I wanted to second your comments about the implications of this kind of scalable technology for MR as an industry. I believe you are right that webcam-based data collection (both facial expression analysis and eye tracking) is going to have a disruptive impact on traditional MR, probably at a scale that lab-based neuromarketing has so far failed to achieve. And I like your emphasis on combining this technology with traditional measures – marketers like evolution, not revolution. Also, your focus on the importance of subgroup comparisons – this is a big challenge for lab-based experiments that rely on pre-defined ANOVA designs.

    These are all compelling capabilities that, along with a low entry-price point, should persuade a lot more marketers and advertisers to give this approach a try. Ultimately, getting beyond trial usage to mainstream research program adoption is going to depend on whether these techniques can produce results that are not just diagnostic, but predictive of marketplace performance. For example, assuming one can “tune” the emotional response profile of an ad (e.g., dampening the sadness, amping up the surprise), what are the benefits … on advertising efficiency, on recall, on purchase behavior, on competitive share, etc. etc? When we start seeing those kinds of results, I suspect we will start seeing a tornado of change sweep thru MR.

  3. Alastair Gordon says:

    September 5th, 2013 at 5:28 pm

    Hi Chris/Steve, thanks for the thoughtful comments. Firstly on the practical uses, if you scroll to the bottom of the post, you’ll see two slideshares, in the second of which I outline several of our ideas for practical marketing uses of this beyond simple ad-testing. I emphasize OUR ideas, what is needed I strongly believe is for marketers to start to capture this data and think of new applications: I see the advent of this as a a huge opportunity for us to re-think our standard approaches, without having to abandon all our traditional questions. Part of this post was in fact to try and move the discussion beyond “ad-testing” and get people thinking more widely. But in terms of ad-testing I think the case is pretty well illustrated already – for instance go to our blog site: http://gordonandmccallum.wordpress.com/ and scroll down to see several case studies, or look up some recent POV’s by Millward Brown on why they are adding it into all their Link tests globally. But to take Chris’ point — this stuff works well (with properly designed studies) on Animatics and Concepts so it does not have to be a finished ad. The more important applications for advertisers and agencies though, probably lie in the ability to use existing material as a basis for analyzing how marketing themes, ideas and styles are working and what is wearing out — using it to proactively design the next wave of advertising rather than just “test” the last! Because you can expose a decent sample of people to a lot of similar material without confusion you have the basis for a revolutionary kind of category and communications analysis that lets you disentangle how people are responding to brand elements, benefits and category imagery. I’ve done some of this now, and it is incredible — but it will get better as we gain experience. The other big communications use I see is quick, massive screening across the globe of new ideas, concepts or campaigns – translation issues are minimized and a lot of material can be assessed quickly.

    Anyway, I’m missing several of your questions I’m sure, but I would take the up the final point from Steve: Yes a relationship to in-market performance will help give comfort (and several agencies and clients are working on this) , but really this is, in my mind a bit of an excuse — with new technologies our industry often applies standards far in excess of those applied to their existing approaches. We know that many survey approaches are biased, and FGD’s results imprecise but we encourage clients to spend millions on them. My view is that facial imaging ALONE, will not be perfectly indicative of in-market performance (just as no survey is now) simply because purchase is driven by many factors beyond immediate emotional response, but that the combination of facial imaging with other methods will indeed be vastly superior to what we are doing now. It’s relatively simple/cheap to try and bluntly researchers should just ‘give it a go’ and see for themselves!

  4. Heval Ceylan says:

    September 7th, 2013 at 3:37 am

    This a great article and I think there is lots of opportunities how this can be used to drive meaningful insights for decision making. I’d love to integrate this into brand&comms tracking and feed it into econometrics with other brand measures to see the impact on sales! Is there anyone already doing this?

  5. Alastair Gordon says:

    September 8th, 2013 at 12:01 am

    Thanks Heval – glad you see the potential. Part of the reasons for this article was to encourage people to think of uses beyond simple “ad-testing” etc. — exactly the sort of thing you mention! At the moment I believe there are a number of big companies quietly trying this kind of technology out in a variety of ways, and tracking is an obvious application, especially in the context of Community or longitudinal panels. Perhaps because most people initially saw this as ‘another variant of neuroscience’, most of the initial work has been about one-off ad or concept tests, but I think this is starting to change as people realise the possibilities of scale. Let me know if you find clients wanting to try a different approach!

  6. Ivan Gruer says:

    September 9th, 2013 at 5:30 am

    Very interesting! Thanks Alastair! Technology apart, as in real life communication, inferences from facial expressions or, in general, from non verbal communication are something that must be checked with questions in order to avoid mislead interpretations.
    The non verbal communication (facial expression, gestures,…) is connected with emotions, however it is only the final result of a inner process that is not possible to understand only with such a analysis since it could be biased by wrong assumptions and interpretations (e.g.: cultural, character divergences).
    Anyhow, in my humble opinion, such a tool might be extreamly useful in order to generate questions to adress in order to better understand feelings and sentiments. I would say that infiring something directly from facial expression is tricky.

  7. Alastair Gordon says:

    September 9th, 2013 at 4:06 pm

    Hi Ivan. Good points – I agree. Emotional response to marketing stimuli reflects and interacts with more deeply held motivations or important social drivers. My view is that any good emotional marketing strategy needs to understand both spontaneous reaction and other factors. But I’d argue that MR has, in recent years, been rather good at identifying deeper personal motivations, but pretty bad at relating those to how well the motivations get translated into marketing material that will create spontaneous cut through. Hence we see lots of promotions, ads etc. that seem pretty similar and dull because everybody is addressing the same motivational “message” without realizing that reaction to seemingly “trivial” elements of delivery matter are often key7 to success. The good thing with facial imaging is that recording spontaneous reaction does not preclude also asking people about motivations or social pressures etc.

    One thing I would say though, based on work I’ve done, is that because with this technology reaction can be precisely related to specific elements across multiple videos or concepts, you can quickly build up a picture of what makes a category tick and where the issues are. So, for instance, if you looked at (e.g.) six typical/contemporary ads for shampoo in the US, it’s possible to feedback to a client what kind of messages/style elements seems to best evoke response in the category, how people react to brand/packaging elements for each brand and what sort of creative approaches are working or not working. This can be sub-analyzed by category usage, demographics etc. So a lot of the kind of information clients currently do quite complex studies to obtain can actually be gathered primarily by ‘watching faces’ if you design the study properly.

  8. Mihkel Jaatma says:

    September 10th, 2013 at 9:17 am

    Fully agree!! And bring on all those validation projects! :) We’ve just again increased our red hot heavily overloaded data science team at Realeyes to deal with the increasing load of business outcome prediction projects.

    Current prediction projects include creative quality as defined by Cannes Lions nominations performance, ad zapping/view-through, likeability, social sharing and purchase intent. Actively sourcing for data partners to cover other business metrics like sales and shop traffic – please do get in touch about that.

    PS! Of course combining survey data with emotions will increase predictive power over either alone. By and large, more and different data sources always strengthen the analysis. The irony is that soon enough it will be the survey data that is the difficult part to obtain. My bet is that the moment we have more high quality emotions data on large audiences than survey information is not far. Couple of years probably.

    Great article Alastair, thank you for putting the word out!

    Chris – large part of our work is on finished pieces on media side – how much to invest behind creative, how to optimize distribution across all options that video agencies have at their disposal these days – very practical stuff, not just nice looking add-on.

  9. Alastair Gordon says:

    September 10th, 2013 at 8:24 pm

    Hi Mihkel, Thanks for this and your comment on the Neuromarketing LinkedIn group. Copying my reply there for Greenbook community:
    ….. glad you are getting into Prediction work – I know nViso is doing work in this area and believe Affectiva is as well, so clearly end-clients are seeing potential in the technology beyond simple one-off testing. Interestingly the nViso team have just done a very large-scale study with medical researchers in Switzerland in the area of using the technology to predict medical outcomes. I think this kind of work underpins the hard-science basis for prediction based on facial expressions and issues of ‘validity’ are now coming up much less than when I started working with this technology 3 years ago.

    I do agree that, particularly for “screening” type applications for specific stimuli we are going to see increased use of facial imaging on it’s own (embedded into a website or in “minimal questions surveys” etc.). For more powerful analytics and relationship to longer-term marketing outcomes though, my feeling is we need to build links to motivational research (to understand deeper needs states etc.) and social drivers of purchase (to understand trade-offs of emotional wants with those of others). The good news is that facial imaging integrates easily with other methods, so we only need more clients willing to think about the benefits of designing new applications and outputs and to give it a go!

Leave a Reply

*

%d bloggers like this: