Download the
NEW e-book from GreenBook: Insights That Work - best research from the GRIT Top 50 companies | DOWNLOAD NOW

Can We Stop Arguing Over What’s The “Best” Methodology?

By Ron Sellers

Blogs and message boards are a great way to discuss research questions.  Some discussions I’ve seen recently:

  • Is it better to do one long survey or multiple shorter surveys?
  • Is qualitative research or quantitative research more likely to result in usable insights?
  • Should researchers be using CATI interviews or online panels?
  • Are focus groups superior to IDIs, or do IDIs provide greater insights?
  • Are traditional focus groups better than online qualitative, or is it the other way around?
  • Are online panels better than social media sampling, or vice versa?

The fascinating thing is that in every case, the posed question is then followed by multiple answers and discussion of which option is right.  People passionately defend their selection and denigrate the other choice.  I’ve even seen some nasty comments and arguments arise from these debates.

The funny thing is that to have a blanket debate like this on these topics is largely pointless.  There simply is no overall answer to whether qualitative projects should be online or face-to-face.  There is no way to answer whether qualitative or quantitative work generates greater insight.  That’s like arguing whether every American would be better off driving a minivan, a family sedan, or a pickup truck.  If you want to haul lumber and tow your boat, you drive a pickup.  If you want to take your six kids to the soccer game, you drive a minivan.

To put it bluntly, saying a minivan is simply a better vehicle than a family sedan is, well, rather dumb.  So is saying IDIs are simply better than focus groups (or the other way ‘round).

As researchers, part of our responsibility is to know the best way of getting the insights we seek, taking into consideration the target audience, the geography, the budget, the timeline, the topic, and the information objectives.  There are times when telephone research is the best option.  There are other times when the project is much more appropriate for an online approach.  And guess what – there are even times when mail surveys or intercept interviews are the preferable way to go.

So can we stop arguing over what’s the “best” methodology?  There is no “best” methodology or approach, other than the fact the “best” methodology is the one that will be most ideal for the specific project at hand.

Unless you really think that everyone in the world should drive a minivan.

Please share...

6 responses to “Can We Stop Arguing Over What’s The “Best” Methodology?

  1. Great post, Ron. I think part of the problem about arguing over what’s “best” is that in newer research spaces (like social and mobile), we don’t have as many established guidelines. We also see a lot of vendors trying to sell us on which method is best. How can we come together as an industry and share our learning about which of these new tools are suited to answer specific research questions?

  2. Sheila, I’m not sure how we can all come together as an industry. Frankly, I don’t see that happening. Too many people are disinterested in learning; too many others only want to promote their own work. That’s not an indictment of the MR industry, but of human nature in general – I have no doubt it’s the same in other industries.

    I think it’s simply up to us as individuals to do what we can. That means if you’re selling something, show some respect for competitive methodologies and don’t trash them as junk. If you’re buying something, let the salespeople know you don’t appreciate hearing how other methodologies are junk and theirs is the only possible solution. Also be open to new approaches while not dismissing the tried-and-true. It’s up to us as individual professionals to work on this issue.

Join the conversation

Sign up to our newsletter

Don't miss out...
Get great insights content delivered straight to your inbox.
I agree to receive emails with insights-related content from GreenBook.


You can manage your email preferences or unsubscribe at any time. GreenBook protects your privacy under the General Data Protection Regulation.