Insights Industry News

August 7, 2014

Rating Scales: A Lack of Drama in 1 Act

It’s easy to ignore research on research in favor of past practices – when people say, “I’m just comfortable with it. So let’s keep it.”

Jeffrey Henning

by Jeffrey Henning

Chief Research Officer at Researchscape

0

speak see hear no evil

 

By Jeffrey Henning

Act 1 – Setting: A Marketing Researcher’s Office

Charlie: How’d that last survey go?

Jessie: Well, the results were pretty clear that consumers preferred Concept C, but it was so different from what the brand had done before that they went with Concept A instead. The brand managers trusted their gut and their past experience more than the research.

Charlie: That’s too bad. Happens sometimes. Well, let’s see about this new questionnaire you submitted.

Jessie: It’s for a new concept test.

Charlie: OK, here you’re using a numeric scale with just the endpoints labeled. According to Krosnick and Tahk in “The Optimal Length of Rating Scales to Maximize Reliability and Validity”, such scales are less consistent, especially with lower educated participants. Let’s use a five-point fully labeled scale, instead. It’s more reliable, according to the research.

Jessie: Well, I always use the 0 to 10 point scale. I’m just more comfortable with it. So let’s keep it.

Charlie: But the research shows…

Jessie: It’s always worked for us in the past. No need to change.

Epilogue

I have a conversation like the above at least once a week. Researchers aren’t so different from the users of our research. It’s easy to ignore research on research in favor of past practices. That’s why I teamed up with QuestionPro to create this video walking people through best practices for interval rating scales. Enjoy!

 

0

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

More from Jeffrey Henning

A Festivus for the Rest of Us Respondents

Research Methodologies

A Festivus for the Rest of Us Respondents

Reflecting on how we can improve survey design for respondents.

Jeffrey Henning

Jeffrey Henning

Chief Research Officer at Researchscape

Aliens vs. Dinosaurs

Brand Strategy

Aliens vs. Dinosaurs

Given the diverse backgrounds of market researchers, there is a real need to continuously train.

Jeffrey Henning

Jeffrey Henning

Chief Research Officer at Researchscape

Researchers and the Love of Learning

Insights Industry News

Researchers and the Love of Learning

MRII’s survey on how the market research industry is doing in career satisfaction, growth opportunities, and learning preferences.

Jeffrey Henning

Jeffrey Henning

Chief Research Officer at Researchscape

#MRX Top 10: The Rise of the Amateur, of AI, and the Death of Expertise

#MRX Top 10: The Rise of the Amateur, of AI, and the Death of Expertise

Jeffrey Henning details the 10 most retweeted links shared using #mrx over the last two weeks.

Jeffrey Henning

Jeffrey Henning

Chief Research Officer at Researchscape

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*