Bias and the Election: What No One is Talking About*

Bias impacts all of us, even pollsters, researchers, and decision makers.

By Katja E. Cahoon

Since the November 2016 election, much has been written and said about the discrepancy between polling results and the actual outcome. But one topic is almost completely ignored, and I personally realized it the hard (or shall I say, unpleasant) way.

There have been important discussions about sample size and sample representativeness, sampling and non-sampling error. About using better methodologies and approaches that bypass the rational mind and what people say (e.g., Implicit Association Testing). About simpler ways of getting data (e.g., text analytics), and accounting for the lack of ability or willingness to answer a question truthfully (e.g., because it might not be politically correct). The latter brings us back to better methodologies and asking non-biased versus biasing questions. Even the challenges around understanding probabilities and different models have been discussed.

During the excellent ARF / Greenbook Election 2016 Debrief – Research & Analytics Event, many of the above topics were discussed by the speakers and panel members[1]. This event provides insightful responses about what went right and wrong. It is also relevant for market research in general, especially given high product failure rates (it is not 80%, that is an urban myth, but it is still high enough[2]).

One topic was mentioned in passing by a few of the knowledgeable speakers and panelists, especially Melanie Courtright, EVP at Research Now, which brings me back to my uncomfortable wake-up call:

Two days after the election, still reeling from the surprise, I had the pleasure of doing an in-depth IDI with a delightful man in his forties from the Mid-West about a topic related to work and finances. He had a Master’s degree, was a manager for a small company, and happily married with two young children. He was the kind of dream participant qualitative researchers hope for – open about his life, finances, fears, and hopes, generous with his time, and both thoughtful and able to discuss his emotions. His main concern, reiterated over and over, was to provide for his children and wife combined with his fears about not being able to do so because of rising health care and living expenses. My heart went out to him. And then he revealed that he voted for Trump.

It is rare to experience undiluted cognitive dissonance, but that is exactly what I felt. For months on end I had considered Trump voters to be largely the confederate flag waving poor white males, the uneducated, the disenfranchised – a frame influenced by the media. This man was none of these things (except white and male). I am grateful for my training as a psychotherapist and my experience as a researcher, which helped me to catch myself. I was able to continue to listen to his challenges with real empathy. After the interview ended, I reflected on my experience and realized: it is not just about having the right data (there were indications that Hillary was not the clear-cut winner most pollsters and publications made her out to be, some indeed correctly called out Trump as the winner[3] as did Michael Moore). It is just as much about being able to see all the data and its implications, as opposed to be being blinded by our own frames, cognitive biases, and media generated impressions.

And that is precisely what is not talked about enough: bias impacts ALL of us, not just voters and consumers, but also pollsters, researchers, and of course decision makers. Melanie Courtright puts it best, “they were all wrong in the same direction, that indicates a bias.” A “democratic bias,” as Raghavan Mayur, President of TechnoMetrica Market Intelligence points out. This bias is reflected in the media. It certainly impacted me; I have a suspicion I am not the only one. This is the “echo chamber.” I should know better as a psychotherapist and highly trained researcher but as the wonderful Daniel Kahneman puts it in Thinking, Fast and Slow, neither intelligence nor experience protects from falling prey to cognitive errors.

And decision makers are impacted as well. Overconfidence in the Clinton camp, especially toward the end, points to that. This is certainly not limited to politics, it happens all the time in business as well. All the way back in the 1970s Irving Janis wrote his ever-relevant book Groupthink. Without being privy to what exactly led to the systematic faulty forecasting and overconfidence I suspect aspects of groupthink played a role (indeed, overconfidence is one of the hallmarks of groupthink).

So, what can we do in addition to becoming better and better at collecting data? It is pretty well accepted at this point that consumers (and especially voters) are not rational beings and are impacted by cognitive errors and biases. But have we – marketers, researchers, and decision makers – truly accepted that we are as well? This is precisely the first step of the three steps of overcoming cognitive bias: true understanding and acceptance that we have them as much as any other human being. A Mistakes Were Made, but not by me kind of attitude does not serve us here. In the next article, I will dive deeper into the three steps of overcoming cognitive bias.

[1] https://youtu.be/MfqGTNG2HeQ

[2] http://sloanreview.mit.edu/article/why-great-new-products-fail/

[3] http://www.reuters.com/article/us-usa-election-polling-industry-idUSKBN136015

*The use of hyperbole is deliberate. Of course, some, but very few, researchers are talking about this explicitly.

You can leave a response, or trackback from your own site.

9 Responses to “Bias and the Election: What No One is Talking About*”

  1. John Holcombe says:

    March 8th, 2017 at 5:53 pm

    Lovely article Katja. You continued to listen with empathy. If there is a way to overcome bias – either personally or for an organization – I would say its empathy. The ability to understand and see others’ worlds, share their feelings and actually appreciate them as people key to unlocking insight, and eliminating bias. To paraphrase Carl Rogers, if you truly listen to people in an empathetic way, you run the risk of changing yourself. That makes people very, very uncomfortable. Most people prefer to follow Argyris’ Ladder of Inference; we assume our assumptions are the same, and selectively filter our experiences and the data we accept so that everything reinforces our existing worldview. The echo chamber has become much easier to sustain in a connected world. Marshall McCluhan said “a point of view (opinion) can be a dangerous luxury when substituted for insight and understanding.” Bias becomes dangerous when groups decide their “point of view” is the only “right” one.

  2. Katja Cahoon says:

    March 13th, 2017 at 11:01 am

    John Holcombe, thank you for your thoughtful and insightful response! I love that you brought up Carl Rogers. When I think of empathy I also think of Brene Brown who has done wonderful work in that area. Again, wonderful to read your post!

  3. Bart Borkosky says:

    March 15th, 2017 at 10:45 am

    Katja, thank you for your honesty. Looking forward to the next article.

  4. Randy Adis says:

    March 15th, 2017 at 12:10 pm

    Excellent article, Katja! A critical and under-explored topic. I’m greatly looking forward to the future installments.

  5. Nicholas Tortorello says:

    March 15th, 2017 at 12:19 pm

    Thank you Katja for a courageous and wonderful article. Thanks also to John Holcombe for his thoughtful and insightful response. As a veteran Survey Researcher and Consultant for over 50 years.(Yes, when I started in the business we were using slide rulers and abacuses to tabulate data). I learned a long time ago that researcher bias was every bit a disaster as a poor sample design, biased questions, and erroneous analysis. I can also tell you when one does legal research and is on the witness stand for over six hours explaining the results to a survey or testifies before Congressional Committees on survey results that the survey must be as white and clean as the driven snow. To my eyes It was easy to see that group think and sheep following the herd was leading the way in the past Presidential election. In all the public and private speeches I gave before the election, I said clearly that “Trump could win”. since polling bias was rampant and even Nate Silver was saying Hillary’s chances of winning were *80%”. Unfortunately, even highly intelligent people tend to believe that people similar to themselves are always right and the only reality.

  6. Katja Cahoon says:

    March 15th, 2017 at 12:54 pm

    Nicholas, thank you so much for your kind response – I very much appreciate your veteran perspective! You raise important points and, yes, neither intelligence nor experience protects from these things – sometimes on the contrary :). It’s hard work overcoming bias, frames, and cognitive errors.

  7. Katja Cahoon says:

    March 16th, 2017 at 11:57 am

    Thank you, Bart! Working on it!
    Randy, hi! Thank you and always nice to hear from you!

  8. Lisa Courtade says:

    March 27th, 2017 at 12:28 pm

    Katja,
    It took me a while to get to this in my in-box but I’m glad I flagged it to read. I’m in vehement agreement. What comes to mind for me is a statement from Jonah Lehrer and Big Thinker Tauriq Moosa that concludes that “smarter people may have a particularly hard time talking themselves out of their biases”. We believe because we know that biases exist, that we cannot possibly fall victim to them.
    Lisa C

  9. Katja Cahoon says:

    March 27th, 2017 at 4:17 pm

    Lisa, wonderful to hear from you! Thank you so much for adding your perspective and Moosa’s quote – I could not agree more. I will be speaking more on this topic at IIEX NA – hope to see you there! Warmly, Katja

Leave a Reply

*