The Fallacies of Facial Exploring Prophecy Feelings vs Facial Coding
By Michael Sankey Ph.D. and Ken Roberts
Forethought welcomes all good science into the world of marketing. We are advocates for evidenced-based decision making in marketing and we welcome competition. But every now and again we see the need to examine alternate offers for our own education and that of our clients.
Recently, we found it interesting and worrisome to learn that more than a third of global Fortune 500 companies are using facial coding for ad pre-testing and some (e.g. Unilever and Mars) have made it a mandatory component for all copy-testing. Brands want to understand emotion in consumption. Great, no quarrel there. So, why does this news worry us? In a nutshell, facial coding for ad testing does not predict purchase behavior.
What are our credentials to comment? Since 2007, we have been in an ongoing conversation with world-leading academics from MIT, Cornell, Duke, ANU, Monash, UNSW and LBS to examine our methodology, Prophecy Feelings (which quantifies and models emotion in choice at a category, brand and communications level), and explore it against other emotions measurement methods such as facial coding. Our post-doctoral colleagues have made it their business to understand the differences and the valid applications of our own approach versus competitor methods. We have been invited to present and debate the validity of emotions measurement at international marketing science conferences including IIEX (USA & Aust), ARF Annual Conference and Audience Measurement (USA), MRMW (USA), AME (China), AMSRS (Aust) and WARC Researching Implicit (UK) conferences. Our work in this space has been rigorously tested in being granted patents in the USA, UK and Australia, during the peer-review process for the Marketing Science Practice Prize and for publishing in the journal of Marketing Science.
We wanted to offer you some of what we have learned:
- Facial coding is based on the work of Ekman, who identified six basic facial expressions of emotion that claimed to be universally recognized. From a marketing perspective, his theory was not grounded in understanding discrete emotions linked to consumption behavior. Conversely, the Forethought Prophecy Feelings scale measures implicit emotional responses to brands and communications on nine discrete emotions. These emotions have been empirically demonstrated to drive consumption behavior across different categories and regions (see Laros & Steenkamp, 2005).Moreover, and most critically, there is minimal, if any, scientific evidence establishing facial coding as a predictor of market performance. The Prophecy Feelings scale is scientifically validated and provides a causal model for the relative importance of nine discrete emotions in driving consumption behavior across categories with high degrees of predictive validity (i.e. correlations frequently above 0.7 in predicting future changes in market share with a one period lag and validated using third-party data). The technique has been published in top tier academic journals for its ability to augment explanatory power in models of choice and delivers far richer insight than commonly employed, stated emotional measures.
- Emotion is not constrained to facial expression – it must also consider body movement, especially for social emotions such as Pride or Shame. Pride and Shame have not been linked to distinct facial behaviors. The Prophecy Feelings scale animates both body movements and facial expressions, thus enabling a more holistic representation of the emotion. Moreover, an emotion such as Pride (which is not captured in facial coding) has frequently been proven in Forethought studies to be a strong driver of consumption behavior across many diverse categories.
- While appropriate for measuring dynamic (second-by-second) content such as video, facial coding is extremely limited in its ability to measure static content (e.g. print, brand logo). In contrast, Prophecy Feelings captures emotional intensity after the stimulus (static or dynamic) has been shown. This data may then be used to model the hierarchy of emotions driving purchase behavior.
- Forethought believes that to most effectively assess communications performance, marketers should be assessing campaigns on their ability to change brand performance (rational and emotional) on the scientifically derived drivers of market share. Facial coding does not measure change in the brand’s emotional performance due to the communication – it simply measures the emotional performance of the communication. In contrast, Prophecy Feelings determines creative efficacy by measuring the degree to which the creative improves/detracts from the brand’s performance on the key emotional consumption drivers; however, it can also be used to assess the emotional performance of the creative itself.From our experience working with clients across many diverse categories, we have seen widespread evidence whereby the creative has elicited strong levels or emotion (positive and/or negative) – however, the same creative did not shift the emotional performance of the brand (i.e. the emotion elicited by the ad was not linked to the brand). If the objective of the communication is to bring about a business outcome such as gaining market share, then the creative measurement should be assessing change in emotional performance of the brand.
- Facial coding is dependent on the experience of relatively strong emotions – muscle movements that occur below the threshold of visual observation cannot be analyzed. Facial coding has greater sensitivity for some emotions (e.g. Happiness) than others (e.g. Surprise and negative emotions such as Contempt). Indeed, only the feeling of Happiness with its expressive feature of the “smile,” is observably related to the underlying physiological and facial pattern of expression (Wolf, 2015). While both facial coding and Prophecy Feelings capture different degrees of emotional intensity, the Prophecy Feelings scale is arguably more sensitive and can detect more subtle emotional differences, as it is not dependent on a physical manifestation of emotion.
- Deploying facial coding outside of a controlled environment is challenging. Three important considerations – pose, illumination, and expression – need to be tightly controlled. For instance, on a mobile device the respondent may be moving around or shift the angle in which they’re holding the device thereby hindering the ability to detect facial expressions. Prophecy Feelings is not subject to the same limitations as the scale captures discrete emotion and contains contextual elements that the emotion pertains to.
- Facial expressions are highly complex and context-dependent. For example, gender, culture and age differences in expressiveness have all been observed. Studies have demonstrated that perceptions of emotions through facial expressions are not universal, but highly influenced by cultural contexts (Gendron, Roberson, Marietta van der Vyver, & Barrett, 2014).
- Facial coding has challenges in accounting for differences in facial morphology. For example, some people have curvature to the mouth that naturally (i.e., when not otherwise emoting) – looks like a smile or a frown.
From the evidence we’ve seen, facial coding does not inform which emotion should be elicited in communications (i.e. it doesn’t help to understand the hierarchy of emotional drivers of market share in a given category), rather it is a moment-by-moment measure of emotional elicitation in creative.
Marketers must first understand which emotion to communicate. This can be achieved by building a quantitative model determining the hierarchy of importance of discrete emotions driving category consumption (note to facial coding: it’s not always Happiness!), then undertake the process of working with research and agency partners to understand how that emotion is best triggered for your brand to help inform communications development. The creative should then be assessed to determine its performance in activating the target emotion, while at the same time understanding how the emotional content may be optimized in both current and future creative.