Can We Stop Arguing Over The “Best” Methodology?

arguingby Ron Sellers

Blogs and message boards are a great way to discuss research questions.  Some discussions I’ve seen recently:

  • Is it better to do one long survey or multiple shorter surveys?
  • Is qualitative research or quantitative research more likely to result in usable insights?
  • Should researchers be using CATI interviews or online panels?
  • Are focus groups superior to IDIs, or do IDIs provide greater insights?
  • Are traditional focus groups better than online qualitative, or is it the other way around?
  • Are online panels better than social media sampling, or vice versa?

The fascinating thing is that in every case, the posed question is then followed by multiple answers and discussion of which option is right.  People passionately defend their selection and denigrate the other choice.  I’ve even seen some nasty comments and arguments arise from these debates.

The funny thing is that to have a blanket debate like this on these topics is largely pointless.

There simply is no overall answer to whether qualitative projects should be online or face-to-face.  There is no way to answer whether qualitative or quantitative work generates greater insight.  That’s like arguing whether every American would be better off driving a minivan, a family sedan, or a pickup truck.  If you want to haul lumber and tow your boat, you drive a pickup.  If you want to take your six kids to the soccer game, you drive a minivan.

To put it bluntly, saying a minivan is simply a better vehicle than a family sedan is, well, rather dumb.  So is saying IDIs are simply better than focus groups (or the other way ‘round).

As researchers, part of our responsibility is to know the best way of getting the insights we seek, taking into consideration the target audience, the geography, the budget, the timeline, the topic, and the information objectives.  There are times when telephone research is the best option.  There are other times when the project is much more appropriate for an online approach.  And guess what – there are even times when mail surveys or intercept interviews are the preferable way to go.

So can we stop arguing over what’s the “best” methodology?  There is no “best” methodology or approach, other than the fact the “best” methodology is the one that will be most ideal for the specific project at hand.

Unless you really think that everyone in the world should drive a minivan.

Originally published at The Green Book Blog
About the Author:
Ron Sellers has 23 years of experience designing and conducting research that makes a difference. Thirteen of those years were spent running Ellison Research, which he acquired in 2008 and rebranded into Grey Matter. Ron’s work has been cited extensively by the international media, including Harvard Business Review, Associated Press, USA Today, Los Angeles Times, Financial Times of London, USA Radio Network, Washington Post, and a host of other media in the U.S., China, Sweden, Russia, New Zealand, Korea, Canada, the Philippines, Norway, Australia, England, and Central America. His work has even been used in panel hearings before the U.S. Senate and quoted in Jay Leno’s opening monologue on The Tonight Show.



  1. […] stuck in the rut of fighting for the right methodology – when we should be fighting about the BEST methodology to get the information we […]

Speak Your Mind