1. Greenbook 2
  2. Greenbook-Mobile-6.29.16-

Can We Stop Arguing Over What’s The “Best” Methodology?

By Ron Sellers

Blogs and message boards are a great way to discuss research questions.  Some discussions I’ve seen recently:

  • Is it better to do one long survey or multiple shorter surveys?
  • Is qualitative research or quantitative research more likely

By Ron Sellers

Blogs and message boards are a great way to discuss research questions.  Some discussions I’ve seen recently:

  • Is it better to do one long survey or multiple shorter surveys?
  • Is qualitative research or quantitative research more likely to result in usable insights?
  • Should researchers be using CATI interviews or online panels?
  • Are focus groups superior to IDIs, or do IDIs provide greater insights?
  • Are traditional focus groups better than online qualitative, or is it the other way around?
  • Are online panels better than social media sampling, or vice versa?

The fascinating thing is that in every case, the posed question is then followed by multiple answers and discussion of which option is right.  People passionately defend their selection and denigrate the other choice.  I’ve even seen some nasty comments and arguments arise from these debates.

The funny thing is that to have a blanket debate like this on these topics is largely pointless.  There simply is no overall answer to whether qualitative projects should be online or face-to-face.  There is no way to answer whether qualitative or quantitative work generates greater insight.  That’s like arguing whether every American would be better off driving a minivan, a family sedan, or a pickup truck.  If you want to haul lumber and tow your boat, you drive a pickup.  If you want to take your six kids to the soccer game, you drive a minivan.

To put it bluntly, saying a minivan is simply a better vehicle than a family sedan is, well, rather dumb.  So is saying IDIs are simply better than focus groups (or the other way ‘round).

As researchers, part of our responsibility is to know the best way of getting the insights we seek, taking into consideration the target audience, the geography, the budget, the timeline, the topic, and the information objectives.  There are times when telephone research is the best option.  There are other times when the project is much more appropriate for an online approach.  And guess what – there are even times when mail surveys or intercept interviews are the preferable way to go.

So can we stop arguing over what’s the “best” methodology?  There is no “best” methodology or approach, other than the fact the “best” methodology is the one that will be most ideal for the specific project at hand.

Unless you really think that everyone in the world should drive a minivan.

Share
You can leave a response, or trackback from your own site.

6 Responses to “Can We Stop Arguing Over What’s The “Best” Methodology?”

  1. Tweets that mention Can We Stop Arguing Over What’s The “Best” Methodology? | GreenBook -- Topsy.com says:

    February 21st, 2011 at 7:52 am

    […] This post was mentioned on Twitter by Bernie Malinoff and Leonard Murphy, Leonard Murphy. Leonard Murphy said: Can We Stop Arguing Over What’s The “Best” Methodology? http://f.ast.ly/qePA3 […]

  2. Tweets that mention Can We Stop Arguing Over What’s The “Best” Methodology? | GreenBook -- Topsy.com says:

    February 22nd, 2011 at 3:30 pm

    […] This post was mentioned on Twitter by Research Dude and JenniferBylok, Kantar Media. Kantar Media said: New blog post Can We Stop Arguing Over What’s The “Best” Methodology? http://bit.ly/hXFVYd (via @lennyism) […]

  3. Sheila Seles says:

    February 24th, 2011 at 9:49 am

    Great post, Ron. I think part of the problem about arguing over what’s “best” is that in newer research spaces (like social and mobile), we don’t have as many established guidelines. We also see a lot of vendors trying to sell us on which method is best. How can we come together as an industry and share our learning about which of these new tools are suited to answer specific research questions?

  4. Ron Sellers says:

    February 28th, 2011 at 7:38 pm

    Sheila, I’m not sure how we can all come together as an industry. Frankly, I don’t see that happening. Too many people are disinterested in learning; too many others only want to promote their own work. That’s not an indictment of the MR industry, but of human nature in general – I have no doubt it’s the same in other industries.

    I think it’s simply up to us as individuals to do what we can. That means if you’re selling something, show some respect for competitive methodologies and don’t trash them as junk. If you’re buying something, let the salespeople know you don’t appreciate hearing how other methodologies are junk and theirs is the only possible solution. Also be open to new approaches while not dismissing the tried-and-true. It’s up to us as individual professionals to work on this issue.

  5. Can We Stop Arguing Over The “Best” Methodology? - Research Access says:

    March 13th, 2013 at 3:22 pm

    […] Originally published at The Green Book Blog About the Author: Ron Sellers has 23 years of experience designing and conducting research that makes a difference. Thirteen of those years were spent running Ellison Research, which he acquired in 2008 and rebranded into Grey Matter. Ron’s work has been cited extensively by the international media, including Harvard Business Review, Associated Press, USA Today, Los Angeles Times, Financial Times of London, USA Radio Network, Washington Post, and a host of other media in the U.S., China, Sweden, Russia, New Zealand, Korea, Canada, the Philippines, Norway, Australia, England, and Central America. His work has even been used in panel hearings before the U.S. Senate and quoted in Jay Leno’s opening monologue on The Tonight Show. […]

  6. Can We Stop Arguing Over The “Best” Methodology? - Research Access says:

    November 17th, 2013 at 5:12 pm

    […] Originally published at The Green Book Blog About the Author: Ron Sellers has 23 years of experience designing and conducting research that makes a difference. Thirteen of those years were spent running Ellison Research, which he acquired in 2008 and rebranded into Grey Matter. Ron’s work has been cited extensively by the international media, including Harvard Business Review, Associated Press, USA Today, Los Angeles Times, Financial Times of London, USA Radio Network, Washington Post, and a host of other media in the U.S., China, Sweden, Russia, New Zealand, Korea, Canada, the Philippines, Norway, Australia, England, and Central America. His work has even been used in panel hearings before the U.S. Senate and quoted in Jay Leno’s opening monologue on The Tonight Show. […]

Leave a Reply

*

%d bloggers like this: