[Editor's Note: The  following post by Ron Sellers was originally published by and is syndicated with permission by The GreenBook Blog.]

I often wonder whether, in research, we spend so much time navigating the complexities of gathering the data that we neglect the all-important field of communicating what we find.  Issues such as online representativeness, phone response rates, and newer forms of data collection (mobile MR, social media sampling, etc.) take up so much of our mental bandwidth that it can be easy to give short shrift to clarity and accuracy in reporting.

One of the biggest and most potentially toxic issues is generalizing.  Marketers dream about homogeneous populations – segments composed of consumers who are all looking to buy a new minivan, or who all have price as the number one criterion when choosing a cell phone provider.  Because of the lure of homogeneity, it’s very tempting to generalize a segment that shows a greater proportion of certain people as being comprised solely of those people.

Geodemographic clustering falls prey to this quite easily.  When I first learned about this technique a couple of decades ago, I was initially quite impressed that companies could identify clusters of people who were all “upscale Caucasians who are early adopters of technology.”  It was a huge disappointment to find out that this segment, rather than being exclusively comprised of these people, simply contained 20% of these people, rather than the 8% who could be found in the general population (I’m making these numbers up).  Although many purveyors of clustering clearly identify their methodology and how the technique is built, I’ve seen how this process is often used by marketers and researchers.  Rather than discuss a cluster with a higher proportion of the desired target, they discuss the cluster as containing nothing but the desired target.
[Read more...]