June 15, 2011

Generally Specific

Why do so many organizations ask broadly general questions and then try to turn them into specifics? As researchers, aren’t we better than that?

Ron Sellers

by Ron Sellers

0

By Ron Sellers

Why do so many organizations ask broadly general questions and then try to turn them into specifics?

This happens constantly in political research. A question might be asked such as, “Do you favor or oppose increasing taxes in order to help reduce the federal deficit?” Although the results might make for some good headlines, this question is so broad as to be meaningless. Increasing which taxes? How much? For which taxpayers? It’s very easy for respondents to toss off an answer such as, “Yes, we need to take difficult measures in order to cut the deficit,” or “No, I don’t support raising taxes.” But give respondents even a shred of detail, and the answers could be entirely different.

Imagine how different the favor/oppose percentages would be to the following options:

• Instituting a national sales tax of 1/3 of 1%
• Instituting a national sales tax of 20%
• Raising income taxes on the wealthiest 1% of Americans
• Raising income taxes on all Americans
• Raising corporate income tax rates
• Raising taxes on tobacco

Yet all of these approaches (and many more) qualify as “increasing taxes.”

I just saw a report from a study claiming that “43 percent of Americans say they would be more likely to buy from a company that ‘manages its business according to Christian principles,’ and only 3 percent were less likely to do so.

What in the world does this even mean? If “Christian principles” means no lying and stealing, it would shock me that only 43% of Americans would prefer to direct their business to that company. If “Christian principles” means hiring only Christians, putting a Bible verse on all products that are sold, and requiring employees to talk to customers about their religious beliefs, there’s no way only 3% of Americans would avoid that type of company. That question is so general as to be meaningless, yet it’s being promoted by the research company and picked up by some media as “news.”

Worse yet, there’s a conclusion to go with the data. “[T]he research shows that the consumer audience is divided between those who favor Christian companies and those who are simply indifferent,” states the report, noting that there’s almost no downside to a company actively promoting Christianity. Let’s find out how well Starbucks or Chevron would do “embracing and promoting” a specific religious stance, and see if there’s no potential downside.

Ask Americans in a survey if they would buy a hybrid vehicle over a gasoline vehicle, and there’s not enough information there for people to answer. Will the hybrid increase gas mileage by 5%, or by 50%? How does the purchase price compare? How about dependability? Styling and comfort? Power? It could be legitimate to ask how likely they are to consider a hybrid based on what’s currently on the market, because respondents have context with which to frame their answers.

Unfortunately, those factors mean more questions, or even detailed techniques such as conjoint. It’s so much quicker and easier just to ask people whether they would want a hybrid version of that current sedan and let them provide a meaningless answer to a meaningless question.

As researchers, aren’t we better than that?

———————————————————————————-

Get Inspired. Stay Informed.  Sign up for our free email newsletter.

0

business leadershippolling

Disclaimer

The views, opinions, data, and methodologies expressed above are those of the contributor(s) and do not necessarily reflect or represent the official policies, positions, or beliefs of Greenbook.

Comments

Comments are moderated to ensure respect towards the author and to prevent spam or self-promotion. Your comment may be edited, rejected, or approved based on these criteria. By commenting, you accept these terms and take responsibility for your contributions.

More from Ron Sellers

Are the Fraudsters More Sophisticated Than the Researchers?

Research Methodologies

Are the Fraudsters More Sophisticated Than the Researchers?

It’s amazing what some people will do in order to make a buck-fifty. Two recent studies have brought to light how sophisticated panel fraud has become...

Ron Sellers

Ron Sellers

Still More Dirty Little Secrets of Online Panels

Research Methodologies

Still More Dirty Little Secrets of Online Panels

Nearly half of your panel data is trash. Here is how to fix it.

Ron Sellers

Ron Sellers

Can Political Polls Really Be Trusted?

Can Political Polls Really Be Trusted?

When political polls fail to predict the exact outcome of an election, maybe they’re not wrong…maybe we are.

Ron Sellers

Ron Sellers

Panel Quality Stinks and Clients Are To Blame

Research Methodologies

Panel Quality Stinks and Clients Are To Blame

Why should panel companies improve their results when clients accept the status quo and won’t pay for better?

Ron Sellers

Ron Sellers

ARTICLES

Moving Away from a Narcissistic Market Research Model

Research Methodologies

Moving Away from a Narcissistic Market Research Model

Why are we still measuring brand loyalty? It isn’t something that naturally comes up with consumers, who rarely think about brand first, if at all. Ma...

Devora Rogers

Devora Rogers

Chief Strategy Officer at Alter Agents

The Stepping Stones of Innovation: Navigating Failure and Empathy with Carol Fitzgerald
Natalie Pusch

Natalie Pusch

Senior Content Producer at Greenbook

Sign Up for
Updates

Get what matters, straight to your inbox.
Curated by top Insight Market experts.

67k+ subscribers

Weekly Newsletter

Greenbook Podcast

Webinars

Event Updates

I agree to receive emails with insights-related content from Greenbook. I understand that I can manage my email preferences or unsubscribe at any time and that Greenbook protects my privacy under the General Data Protection Regulation.*