Insights Industry News

July 31, 2012

The Dangers Of Assuming The Status Quo Is Right

Research needs to consider what new technologies add to the mix, rather than try to fit them to some outdated model of what research is.

Ray Poynter

by Ray Poynter

Managing Director at The Future Place

0

Editor’s Note: In his usual erudite fashion, Ray Poynter challenges us to question new techniques and approaches but also to do the same to our own assumptions about what is the “right way” to do things in MR. It’s a great reminder for us all to be mindful of our own biases and preconceptions regardless of where you are on the “new vs. traditional” research spectrum. We should let the data lead us to conclusions, not our preconceptions, right?

By Ray Poynter

One of the benefits of having spent over thirty years in the market research industry is seeing the same arguments come round every so often. The first time one sees a particular argument it seems novel and convincing, but by the second or third time round one starts to becomes accustomed to it and to be better able to put it in context.

When telephone interviewing (often referred to as CATI – computer aided telephone interviewing) came on the scene in the 1980s people were concerned about whether it would introduce biases due to sampling issues and/or modality effects. These were valid concerns, but they were usually addressed by running comparative studies with face-to-face studies, as if face-to-face were a reliable indicator of ‘truth’.

Instead of asking ‘Is CATI the best available system?’ or ‘Is CATI good enough?’ or perhaps ‘What is CATI good for?’ the typical question that was asked was one almost guaranteed to make CATI look like a second-rate compromise, adopted because it was cheaper and faster. That badly formed question was ‘Is CATI biased?’. Yes, of course it was, everything that involves people measuring people is biased (as I explore in a recent VCU paper titled ‘Are Community Panels Biased?’). If research is to move ahead new techniques should be held to the same standard of inquiry and evaluation as existing techniques. They should not be held to some higher standard, predicated on an implicit assumption that what is already happening is perfect and that new techniques need to produce the same results as a necessary minimum standard of acceptability.

When online research appeared, when online access panels appeared, and when research communities came on the scene, the same poorly constructed questions were asked ‘Is it fully representative?’ and ‘Is it biased?’, as if the other research alternatives were somehow beyond question.

In the last few years researchers have become increasingly aware that our status quo assumptions of rational respondents, being reached representatively, and answering in an informed way are often wide of the mark. Neuroscience, behavioral economics, theories of social behavior, and predictive markets have all shown that market researchers need to re-address the status quo as well as properly addressing emerging ideas.

Research is on the edge of a revolution in the way it conducts its business. Key changes include: research communities, mobile devices, social media, and Big Data. Research needs to work out what these new strands can add to the mix, rather than trying to fit them to some hypothetical and outdated model of what research is and how it should be evaluated.

Key questions for research to address are:

    • 1. What approaches work, for what situations, and for which contexts?
    • 2. For any specific approach, what it strengths and its limitations?
    • 3. For a given business decision, what is the right trade-off between accuracy, speed, money, and depth? (We used to talk about quality, speed, and money – but with many research techniques the quality issue needs to be subdivided into accuracy and depth.)
    • 4. How do we ensure that research buyers are in a position to make informed choices between the various options?

0

emerging technologiesinnovation

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

More from Ray Poynter

Word Cloud Plus Extracts Hot Buzz Topics
Ray Poynter
Word Cloud Plus Extracts Hot Buzz Topics

The Future Place

Word Cloud Plus Extracts Hot Buzz Topics

The GRIT data collected toward the end of 2022 included 918 open-ended replies to the question “Related to insights, research, or analytics, which topics do you follow most closely and…

By Ray Poynter

Managing Director at The Future Place

Observational Data Has Problems. Are Researchers Aware of Them?

Research Methodologies

Observational Data Has Problems. Are Researchers Aware of Them?

Observational data is a tempting shortcut for insights but researchers must consider its potential shortfalls

Ray Poynter

Ray Poynter

Managing Director at The Future Place

Want it Faster, Cheaper, Better? Then You’d Better Sign Up to Prescriptive Analytics

Business-to-Business (B2B) Market Research

Want it Faster, Cheaper, Better? Then You’d Better Sign Up to Prescriptive Analytics

Using Prescriptive Analytics to navigate the data-filled world.

Ray Poynter

Ray Poynter

Managing Director at The Future Place

Company Size Matters: The Differential Impact of COVID-19

Company Size Matters: The Differential Impact of COVID-19

A deep dive into the very different experiences of the MRX community during COVID-19.

Ray Poynter

Ray Poynter

Managing Director at The Future Place

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*