Generalizing: The Bane of Insights

September 29, 2011 By Leave a Comment

stereotype[Editor's Note: The  following post by Ron Sellers was originally published by and is syndicated with permission by The GreenBook Blog.]

I often wonder whether, in research, we spend so much time navigating the complexities of gathering the data that we neglect the all-important field of communicating what we find.  Issues such as online representativeness, phone response rates, and newer forms of data collection (mobile MR, social media sampling, etc.) take up so much of our mental bandwidth that it can be easy to give short shrift to clarity and accuracy in reporting.

One of the biggest and most potentially toxic issues is generalizing.  Marketers dream about homogeneous populations – segments composed of consumers who are all looking to buy a new minivan, or who all have price as the number one criterion when choosing a cell phone provider.  Because of the lure of homogeneity, it’s very tempting to generalize a segment that shows a greater proportion of certain people as being comprised solely of those people.

Geodemographic clustering falls prey to this quite easily.  When I first learned about this technique a couple of decades ago, I was initially quite impressed that companies could identify clusters of people who were all “upscale Caucasians who are early adopters of technology.”  It was a huge disappointment to find out that this segment, rather than being exclusively comprised of these people, simply contained 20% of these people, rather than the 8% who could be found in the general population (I’m making these numbers up).  Although many purveyors of clustering clearly identify their methodology and how the technique is built, I’ve seen how this process is often used by marketers and researchers.  Rather than discuss a cluster with a higher proportion of the desired target, they discuss the cluster as containing nothing but the desired target.
[Read more...]

Filed Under: Market Research Tagged With: , , ,

The Great Data Privacy Debate: A Summary

August 30, 2011 By 2 Comments

[Editor's Note: The  following post by Tamara Barber was originally published by and is syndicated with permission by The GreenBook Blog.]

Recently, Andrew Jeavons, EVP at Survey Analytics, moderated a fascinating live debate on data privacy in market research. The virtual debate has been brewing in blogs, LinkedIn, and Twitter for some time, but a discussion paper on the topic from the MRS and new draft guidelines from CASRO brought this issue back to the forefront in recent weeks.

This webinar featured leaders from each of the three major market research associations, as well as industry opinion leaders on the topic. Here is what the six participants had to say in their opening statements (Note that I’ve remained as true to the dialogue as I could given how fast I can type. I’ve included quote marks where I’m sure of their accuracy):
[Read more...]

Filed Under: Events, Market Research Tagged With: , , , , , ,

Optimizing International Research

August 3, 2011 By Leave a Comment

[Editor's Note: The  following post by Michael Wolfe was originally published by and is syndicated with permission by The GreenBook Blog.]

If you’re a client-side researcher working for a multinational company, much of your time is likely spent on multi-country studies. In the quantitative arena, the advent of online panels has changed much here over the past few years, with the likes of Toluna (access to 4 million individuals across 34 countries – http://bit.ly/noVojq); SSI (respondents across 72 nations – http://bit.ly/ooZgBj) and Lightspeed (http://bit.ly/eEoiMO)to name but a few, offering relatively easy and cost-effective access to large multi-country online panels.

The ability to tap into such globally structured and professionally managed panels has lead, in my view, to many full-service Agency suppliers across the globe confidently confirming their ability to conduct multi-country research.

I wonder if this sense of confidence is always well placed. Doing anything across cultures to a high level, including research, is in my view a complex undertaking and requires a high degree of intercultural sensitivity if genuine insights are to be accessed. This goes well beyond offering a great translation service.

Here are some thoughts on how to optimize international research:
[Read more...]

Filed Under: Market Research Tagged With: , , , , ,

The Dilemma of Social Media Measurement

July 19, 2011 By 1 Comment

[Editor's Note: The  following post by Michael Wolfe was originally published by and is syndicated with permission by The GreenBook Blog.]

Everyone with any sense of what is going on in the field of marketing right now knows that there is almost a mad frenzy about social media, the likes of which we have seldom seen. In fact, this frenzy has become so strong that it has spawned almost a gazillion self-proclaimed experts and gurus. It almost feels like the American Wild West, with the itinerant snake-oil salesman coming to town to sell us a bottle of his miracle cures.

As I have reviewed the literature on this subject, there appears to be three camps or approaches towards social media measurement. These are as follows:
[Read more...]

Filed Under: Market Research, Social Media Tagged With: , , , , , , ,

Market Research Innovation Challenges

June 28, 2011 By Leave a Comment

[Editor's Note: The  following post by Navin Williams was originally published by and is syndicated with permission by The GreenBook Blog.]

A few days ago I read the blog post by Tom Anderson covering the MRIA annual event in Canada.  It seemed like a very lively discussion and wish I had been there!  There are a lot of interesting conferences happening in the MR community (many of which I am missing!) about change and the direction MR is taking.

From Tom’s blog two quotes struck me and I have them verbatim below:

  1. “it’s telling that many now seem to understand that probably well over half of the ‘research’ firms out there won’t be around 5-10 years from now”
  2. “how the biggest firms in our industry have stifled innovation because it has been in their best interest to do so”

Though both the above lines were separated by more than a third of the post, I found them interrelated in the sense that the first is pretty much driven by the second.  The question is who will survive? Will it be the Goliaths of market research who seem reluctant to change the old ways or the new age firm David’s with radically different approaches in their DNA?  In an ideal world it should be a bit of both. The big firms lead with innovation and therefore lead the industry and a bunch of smaller firms cut through the clutter. Is that happening? Maybe a little. Is enough happening? Probably not.

This is not to say all the big agencies are not thinking ahead as some of them are and have been. Many are radically changing to embrace the coming future. Few agencies actually have or have started innovation divisions & innovation officers/executives whose whole brief is to think ahead of the curve.

Below I have tried to explain some of the reasons why I think the industry (as a whole – not individual firms) is where it is:
[Read more...]

Filed Under: Innovation, Market Research Tagged With: , ,

How Do You Assure Online Panel Quality?

June 2, 2011 By 1 Comment

[Editor's Note: The  following post by Ron Sellers was originally published by and is syndicated with permission by The GreenBook Blog.]

In a meeting recently, I was asked a very cogent and interesting question: “How do you assure the quality of online panel sample?”

The answer is very simple: I don’t.

I can’t be positive that the panel sample I am using is good quality. There is no one statistical test or software that can tell you whether you have a reliable sample or not.

Dealing with Internet research panels is like securing your house when you’re going away on vacation. There are all sorts of steps you can take to burglar-proof your home, but the plain fact is that if a burglar wants in badly enough, you’re going to get burglarized in spite of all your careful precautions. Police officers will tell you that the goal of burglar-proofing your home is not to make it impossible for burglars to break in, but to make it much more difficult for them, and therefore much less likely to happen.

Quality checks on Internet panels work the same way. I may not be able to ensure panel quality 100%, but there are plenty of steps I can take (and have taken) to make it a far higher likelihood that I’m getting good data.
[Read more...]

Filed Under: Market Research Tagged With: , , , ,

The Game Mechanics of Social Media

May 18, 2011 By Leave a Comment

[Editor's Note: We continue our on-going series on gaming and research this month with the following post by Jon Puleston; originally published and syndicated with permission by The GreenBook Blog.]

By Jon Puleston

I have been studying game theory deeply over the last few weeks in my quest to work out how to effectively gamify surveys and I have started to see game play mechanics embedded into all sort of activities.

Just for a bit of fun, well as an invented game in-fact if you look at it this way,  I tried to identify and compare the game play mechanics embedded into Twitter, Facebook and LinkedIn to work out which was the best game.

These are some of the basic mechanics of successful games:

  1. Strict rules
  2. A reward mechanism that delivers both pleasure and emotional satisfaction
  3. A well balanced success/failure ratio – An accomplishable challenge
  4. A broad learning curve
  5. A wider competitive element
  6. A balance of luck v skill
  7. Levels
  8. Addiction factor – is it an absorbing activity in itself
  9. Team play

How they score…
[Read more...]

Filed Under: Featured, Gamification, Social Media Tagged With: , , , , ,

Can Qualitative Research Be As Smart As Your Phone?

March 8, 2011 By Leave a Comment

By John Williamson, CEO & Founder, Qualvu

The breakneck pace at which smartphones are being adopted is arguably the biggest story in the history of technological diffusion, and it’s a story that has significant cost, quality and speed implications for market researchers.

Four years ago, Apple sold almost two million iPhone 4’s in its first three days on the market. To date, they’ve sold 57 million. Last year, more than 300 million mobile phones were sold in the first quarter alone. Fifty-four million of those were smartphones – phones capable of running a variety of applications and featuring integrated browsers to connect to the Internet. Google’s Android operating system is growing 866% year-over-year, with more than 160,000 new Android phones activated every day. Worldwide, five billion persons – more than 70% of the world’s population – have mobile phones.
[Read more...]

Filed Under: Innovation, Market Research, Mobile Tagged With: , , , , , ,

Can We Stop Arguing Over The “Best” Methodology?

February 25, 2011 By 1 Comment

arguingby Ron Sellers

Blogs and message boards are a great way to discuss research questions.  Some discussions I’ve seen recently:

  • Is it better to do one long survey or multiple shorter surveys?
  • Is qualitative research or quantitative research more likely to result in usable insights?
  • Should researchers be using CATI interviews or online panels?
  • Are focus groups superior to IDIs, or do IDIs provide greater insights?
  • Are traditional focus groups better than online qualitative, or is it the other way around?
  • Are online panels better than social media sampling, or vice versa?

The fascinating thing is that in every case, the posed question is then followed by multiple answers and discussion of which option is right.  People passionately defend their selection and denigrate the other choice.  I’ve even seen some nasty comments and arguments arise from these debates.

The funny thing is that to have a blanket debate like this on these topics is largely pointless.
[Read more...]

Filed Under: Market Research Tagged With: , ,