Voice Of The Customer Surveys –  Useful Or Fundamentally Flawed?

VOC programmes are highly likely to generate unreliable and misleading results which contribute to an illusion of understanding customers.

clientsurvey

 

By Neal Cole

Voice of the Customer (VOC) surveys are a popular method for organisations to monitor customer preferences and levels of satisfaction. However, how robust is the standard process of interviewing a few customers and then developing and administering a highly structured survey?

The VOC survey framework appears to be based upon an outdated and false view of people as rational, independent agents with relatively fixed preferences. Because of the way the human brain works people have limited access to the emotional, social and psychological motivations that drive much of our behaviour.

 

image

 

Surveys also ignore the power of contextual influences which means that VOC programmes are highly likely to generate unreliable and misleading results which contribute to an illusion of understanding customers.

A flawed measure:

  • Asking direct questions is unreliable because many of our daily decisions are made without much conscious thought and we post-rationalize decisions to ensure consistency with our internal model of the world. When people answer direct questions they attempt to rationalise their decisions and behaviour. They naturally construct a narrative that explains their  actions in a rational and consistent way.
  • However, this fails to capture many of the underlying motivations of human behaviour. Further, as our brains use mental short-cuts to save energy and speed up decision making, we are prone to making sub-optimal and often irrational decisions. This is contrary to how we like to perceive ourselves and naturally we don’t articulate this when answering survey questions.
  • Asking people about importance is especially misleading. Psychologists have found that relative importance of an item is heavily influenced by the ease with which we can retrieve it from our memory. This often correlates heavily with the amount of coverage an issue gets in the media. I noticed this when I managed a survey of financial advisers. The importance of financial strength always shot up after media coverage of any kind of financial crisis.
  • Our ability to recall an event is also limited as a memory is constructed from a series of brief fleeting moments from an experience. We construct a memory from visual snapshots, thoughts, feelings, smells and sounds. The perception of an experience is also influenced by range of factors including our mood, social context, our vocabulary, the physical environment and our knowledge.
  • However, the reconstruction process itself is also the product of these same factors. This means that each time we recall an event we have to piece these elements back together and inevitably our memory changes each time we recall an experience. Indeed, we frequently have false memories of events that we will passionately defend as our brain attempts to retain our internal consistency with our values and beliefs.
  • Our memory is also biased towards  remembering what happened during the most intense moment of an experience, and what occurred at the end of the episode. The length of the experience appears to have little impact on our overall satisfaction with an event.
  • Our herd instinct means we are heavily influenced by what other people in our network and beyond are doing. We are constantly copying people, especially when we find ourselves in a new or uncertain situation. Social norms and trends are also powerful forces, but again we may not  be consciously aware of these influences and so we cannot expect people to articulate this via direct questioning.

Analysis and Action:

VOC surveys also fail to deliver when it comes to analysis of results as little or no allowance is often made for the fundamental flaw in using direct questions to obtain the data.

  • Due to the survey design operational and customer facing areas often find VOC feedback to be too generic and therefore not actionable. They want to drill down to specific locations or touch points but sample size are usually not sufficient to allow for such analysis.
  • Unfortunately research executives can sometimes be pressured into providing analysis based upon tiny sample sizes. This gets quickly circulated around the organization to support changes in service or product delivery. The fact that responses are unlikely to be an accurate reflection of real customer motivations and preferences is usually ignored until results contradict what a senior stakeholder wants the data to say.
  • Sponsors like to set targets to improve VOC scores but ratings tend to fluctuate for no obvious reason. When expensive changes to the product or service are implemented is it surprising that we may not see any change in customer satisfaction?
  • Some organisations now base changes on an understanding of our selected memory bias by focussing on the peak and end moments of an interaction. This may lead to improved VOC scores but does this really mean that we are delivering an improved customer experience?  Are we not just kidding ourselves by using human psychology to temporarily boost a flawed score?

Experiments and Observation:

Top retailers have known for decades that if you want to find out if something new works running a controlled test in a number of stores is more reliable than asking people direct questions. This was the original A/B test and is the reason why online retailers are now some of the biggest users of this experimental research design.

“Just following consumer wishes leads to replaceable products, copycat advertising, and stagnating markets.”
Stephen Brown, Professor at the Kellogg School of Management

Direct questioning of customers is limited for the reasons given above. However, VOC surveys are particularly problematic because the standard framework developed as part of the Six Sigma methodology gives it an aura of validity that the technique does not merit. This is an illusion which Six Sigma followers espouse due a lack of understanding of basic human decision-making and psychology.

More emphasis on conducting experiments, together with observing or listening to real customer interactions would be more effective methods of research. Co-creation can also be a powerful approach to allow brands to open up a two way conversation with their customers. Direct questioning for VOC programs needs to be used sparingly and the results treated with extreme caution.

Thank you for reading my post and I hope you will find some of my other post of interest.

Recommended reading:

imageConsumerology: The Truth about Consumers and the Psychology of Shopping (new revised edition, including a new preface from the author)

You can leave a response, or trackback from your own site.

4 Responses to “Voice Of The Customer Surveys –  Useful Or Fundamentally Flawed?”

  1. Horst Feldhaeuser says:

    March 4th, 2015 at 4:27 pm

    Hi Neale,

    I think it’s crucial to define VoC first. Is it:

    a) A continuous ongoing measure (ideally live) of customer experiences to track specific KPIs. I.e. short CE questionnaires that give a good operational view. When combined with live reporting, these are very useful both operationally and when looking at the bigger picture even strategically. We’ve seen many companies that massively improved their customer experiences and in the way changed the culture of their business through these live CE programs
    b) NPD Research to find out what products & services they need, would pay for etc. As per Ulwick’s process, here we need to look at ‘jobs’ people are trying to get done, ‘outcomes’ they are trying to achieve and ‘constraints’ that stand in their way. Then it’s our (the brand marketer’s) job to come up with the best solution. And yes, please don’t ask them whether they want a faster horse.

    These are two fundamentally different things.

  2. Chris Robinson says:

    March 10th, 2015 at 4:50 am

    Surely all this objectivity and flawed thinking also comes into play when consumers consider at that precise moment to buy a brand. Why would it be any different? Are you suggesting that consumer decision making operates on some super level when it comes to brand consideration? That there is some process that reflects and analyzes the real reasons behind purchase behaviour.? Why would that be the case? We build all our lives, all our relationships around the same flawed processes and we somehow seem to get along just fine. Does marketing demand any finer insights than people lack objectivity but seem to somehow make brand decisions that tend to be consistent on most occasions. I am not sure if this so called complexity is all that relevant. The process between A Interest and B Decision may be complex but it doesn’t require a lot of sophistication to know that people move from A to B despite the fact they cannot actually tell you deep down why? This is all starting to sound like Ernst Dichter’s analysis that postulated cigar smoking was all to do with some oral fixation from breast feeding. Did that ever help sell cigars? I think probably not, even if it were true. Of course the cigar smoker may just say they like the taste. No that cannot be right! Too simple.

  3. Kevin Gray says:

    March 15th, 2015 at 8:48 pm

    The claim that marketing researchers view people as rational, independent agents with relatively fixed preferences is a red herring, and an old one. That said, much VOC and other consumers survey research is amateurish in design, analysis and interpretation. The fundamental problem is not with the methodology itself.

  4. Neal Cole says:

    March 28th, 2015 at 6:15 am

    Thanks for your comments on my post.

    I agree there are some good practices for Voice Of the Customer programmes. Horst, I think your point about changing the business culture is key as this is critical for VOC programmes to really improve the customer experience. Unfortunately I have worked on the client-side and seen a number of examples where the management expected to see improvements in the customer experience without grasping the issue of cultural change and putting the customer at the heart of the business decision making. SixSigma teams also focussed on statistical robustness rather than developing insights from real customer experiences.

    I have written a new post on my blog which outlines strategies for improving the value from VOC programmes which you may find of interest. http://conversion-uplift.co.uk/post/114491976162/improving-voice-of-the-customer-programmes

    Regards

    Neal

Leave a Reply

*