Behavioral Science

September 13, 2020

Honesty of Responses: The 7 Factors at Play

30% of survey respondents lie. Here is why.

Honesty of Responses: The 7 Factors at Play
Jon Puleston

by Jon Puleston

Vice President - Innovation at Lightspeed Research

0

I read this very interesting post by Edward Appleton about the authenticity of people’s online behavior. The authenticity of online respondents is a very interesting philosophical question and is an area we have been looking at recently too – albeit perhaps in a more literal way…

We have been examining ‘honesty’ within the online survey environment, by comparing answers to questions where we hold known norm figures about behavioral activity and building up a picture of levels of honest answering to different types of questions.

We have found that in the main most people in western* markets who respond to surveys are, on the whole, very honest in the answers they give.

Rolled up into averages, about 95% give “honest” answers to the average question – so it is not something you need to worry too much about for a typical survey.

But that figure of 95% is for the average question. This figure can vary dramatically and we have witnessed questions where up to 30% of respondents show signs of untruthfulness based upon the context of how you ask the question and the type of questions being answered and who you are asking.

The reasons for this, though, are complicated. We have identified a range of driving factors that determine the level of authenticity to an answer we may give, these are:

 

The Peacock syndrome

We like to show off and make others think we are smarter, wealthier, happier, better than other people. You can see this issue in effect on magazine readership studies where, for example, the claimed number of readers per copy of Vogue magazine is 12 – about four times higher than the average magazine. The number of people who claim to shop at Harrods the premium London on the TGI survey is about 2 times higher than the estimated footfall*. So you can never really trust stated figures about any claimed behavioral activity if it is socially desirable.

You must also remember that this process works in reverse. So as an example, people admitting to watching soap operas, smoking, or noticing advertising, all things that we don’t like to admit to doing, are all downgraded by us naturally.

*These figures themselves may not be completely truthful, they are plucked from my memory which is a well-known source of untruthfulness if I am to be brutally honest!

 

Pleasers:

This is perhaps more of a cultural issue. , Some people like to answer in certain ways to please you. Now, this is different from the peacock syndrome and it affects different types of questions, those that are less commercial questions and more social questions such as political opinions as an example. We are often very reticent to give our real views on certain topics for fear of perhaps offending. Or we may worry that our views are not socially acceptable. Or even in the extreme case of say, the fear of persecution if you were researching an oppressive political regime.

 

Gamers

Now, these are a set of people who adapt their answers to play the system, which is a big problem particularly in certain countries and for certain types of topics. For example, if I am trying to sample top-end business people and I am offering a high incentive to take part then there will be those who will try to pretend to be business people so that they can qualify to participate in the surveys. But this is not just limited to a small group of fraudulent respondents. A lot of respondents start gaming the system when they simply get bored. They stop saying they do things or buy things to avoid having to answer extra follow-on questions.

 

Disengagement

This leads to the last group which is the disengaged respondents who don’t answer truthfully because they can’t be bothered to think about it. It manifests itself on lots of different levels. Ad recall can vary by upwards of 100% between engaged and disengage respondent groups.

 

We lie to ourselves

We are all naturally quite self-delusional. The image we have of ourselves photographically, for example, is often nothing like the image that other people see. So if you asked me to pick a picture which is a good likeness of yourself nine times out of ten it will be a different picture from what someone else will pick. It results in peacock answering but also a lot of delusional responses to social behavior questions e.g. are you the sort of person who would recycle? We all like to think we do and a lot of us believe we do, but it is something we don’t actually do in practice.

Here is a link to a wonderful paper exploring this issue http://www.tandfonline.com/doi/pdf/10.1080/15298868.2011.617886.

 

The power of the unconscious

So much of what we do is controlled by unconscious decision-making paths. The conscious v unconscious mind is often in conflict. We, as a result, often make irrational decisions and so we are not very good observers of aspects of our behavior which is controlled by the unconscious. As a result, we are hopeless predictors of certain things we do. Obviously, this issue sits at the heart of behavioral economics.

 

General Ignorance

We simply don’t know or don’t remember answers to certain often very clearly obvious questions: do I have brown eyes? I don’t know. Where do I live in the US – 10% of people don’t know the answer to this, particularly if you show them a map of where they live (to the extent we had to stop using maps for this question in our surveys!). We also are unable to predict our future behavior because we don’t really know what we will do. , We are, in fact, particularly bad at this. You may as well pluck a random number out of your head instead of asking somebody if they are likely to buy a new product in the future or not as it is likely to be just as accurate.

 

Cross-cultural differences in truthfulness

I have starred *Western markets earlier on because in non-western markets the level of “honest” answering can drop significantly down to, in some cases, around 70%.

In fact in some countries, you can forget any other worry about panel quality and demographic sample biases. The level of honesty is the Number One issue that you need to contend with. The variance in levels of truthfulness is compounded by factors such as income. If the incentive to take part in a survey is relatively higher in one country v another you will see higher numbers of people motivated to be gamers for example.

 

The big questions

The big questions are to what extent can these relative factors be quantified? What impact do these things have relatively? and what tools and techniques do we have to counter them?

 

Photo by Jametlene Reskp on Unsplash

 

0

behavioral sciencerespondentsurveys

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

More from Jon Puleston

GRIT Sneak Peek: The Top Challenges & Opportunities in Research

Artificial Intelligence and Machine Learning

GRIT Sneak Peek: The Top Challenges & Opportunities in Research

A sneak peek at the Top Challenges & Opportunities section of the upcoming Q3-Q4 2017 GRIT Report.

Jon Puleston

Jon Puleston

Vice President - Innovation at Lightspeed Research

Insights Industry News

Information Addiction: The Implications for Society, Brands, Advertising & Research

Jon Puleston confesses his information addiction, and analyzes the implications it has for society and the commercial arena.

Jon Puleston

Jon Puleston

Vice President - Innovation at Lightspeed Research

Research Technology (ResTech)

The Game Mechanics of Social Media

Comparing the game play mechanics embedded into Twitter, Facebook and LinkedIn to work out which was the best game.

Jon Puleston

Jon Puleston

Vice President - Innovation at Lightspeed Research

Research Methodologies

Questionnaire Piloting: A User Guide

What can you use piloting to do, and how do you go about piloting? Here are some tips and suggestions.

Jon Puleston

Jon Puleston

Vice President - Innovation at Lightspeed Research

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*