Generalizing: The Bane of Insights

stereotype[Editor's Note: The  following post by Ron Sellers was originally published by and is syndicated with permission by The GreenBook Blog.]

I often wonder whether, in research, we spend so much time navigating the complexities of gathering the data that we neglect the all-important field of communicating what we find.  Issues such as online representativeness, phone response rates, and newer forms of data collection (mobile MR, social media sampling, etc.) take up so much of our mental bandwidth that it can be easy to give short shrift to clarity and accuracy in reporting.

One of the biggest and most potentially toxic issues is generalizing.  Marketers dream about homogeneous populations – segments composed of consumers who are all looking to buy a new minivan, or who all have price as the number one criterion when choosing a cell phone provider.  Because of the lure of homogeneity, it’s very tempting to generalize a segment that shows a greater proportion of certain people as being comprised solely of those people.

Geodemographic clustering falls prey to this quite easily.  When I first learned about this technique a couple of decades ago, I was initially quite impressed that companies could identify clusters of people who were all “upscale Caucasians who are early adopters of technology.”  It was a huge disappointment to find out that this segment, rather than being exclusively comprised of these people, simply contained 20% of these people, rather than the 8% who could be found in the general population (I’m making these numbers up).  Although many purveyors of clustering clearly identify their methodology and how the technique is built, I’ve seen how this process is often used by marketers and researchers.  Rather than discuss a cluster with a higher proportion of the desired target, they discuss the cluster as containing nothing but the desired target.
[Read more...]

Short and Very Dry, Please

FoamI like my coffee every day and prefer cappuccinos that are short and dry. Now, for the uninitiated, let me explain. That means a cappuccino with less milk and proportionally more foam. There is no exact measure for what the proportion should be and that is where the skills of a good barista come into play.

But I digress.

I was in a corporate cafeteria this morning and ordered my usual at a Starbucks stand – except that knowing how things are done there, I asked them to make it ‘extra dry’ just to be sure there was no ambiguity.

Sure enough, when I got my coffee, it was far from dry. Actually it was wet enough to be classified as a small latte in some parts. Now this is not completely surprising since as mentioned earlier, this is an art and not a science. Normally it’s a matter of requesting the barista to just make it drier, a request that is usually complied with a smile.

Not here, not today. The barista got into an argument with me debating what dry and extra dry is and should be. At one point, it degenerated into her mocking me sarcastically and wondering if I was in the habit of making my own special cappuccinos every afternoon. And this was not in jest, and there was neither contrition nor acknowledgment that a customer could 1) have a preference and 2) sometimes be right especially when there is no ‘right’ answer.

Fortunately, the café manager happened to be nearby and helped to smooth things over by offering me free coffee which I didn’t accept to make a point (though I’m quite fond of free in general). The barista had quickly and seriously harmed the reputation of that particular coffee stand and turned someone that could have been a loyal customer into someone that well, wrote this blog.

We are reminded of the importance of great customer service everyday – but what a shame that companies seem to invest less time and money in training employees enough in that regard. This is truly an investment, and not an expense.

The Physical and The Digital

Marketing Punters and Pundits alike make stupid predictions.  One of the most laughable of late was the prediction of the early demise of “physical” Marketing- it will all be digital, in the ether, non-corporeal the big-thinkers said.

And like the seers of a “paperless office,” they were dead wrong.

Case in point is a superb piece of Marketing that just crossed my desk today.  The sexy brand Ferrari built their website on Microsoft’s Sharepoint for Internet Sites (FIS) platform, as have thousands of other companies.  Seeing an opportunity, Austin-based Catapult Systems decided to build a Practice out of planning, developing and maintaining websites built on FIS; in order to drive (excuse the pun) the business forward, they decided to do a Marketing campaign leveraging the Ferrari brand in a smart, fun way.

They sent a physical mailer with a nice Ferrari keychain attached.  A VIN number was etched on the keychain.  With simple text, the physical mailer drove (there we go again) folks to a website where one can watch a very simple-yet-compelling video about Sharepoint FIS and how Catapult systems can help companies engage their customers with the Sharepoint technology.  After watching the video, one had to register (with only a few information fields.)

And guess what?  No matter what happens, you get to keep the keychain (and who doesn’t love Ferrari products!)

In one fell swoop, the consumer got a very compelling digital and physical experience.

Now, I’m not trying to stump for Catapult; that’s not my intent or the intent of this blog.  But I do want to heap praise where praise is due.

But more than that, I want to share a bias with you all- I still love physical marketing.  A good giveaway is as compelling as any website.

So throwaway the bathwater but keep that gorgeous and gurgling baby!

[Editor's Note: This post originally appeared on our sister site, MarketingAccess.com]

A New Approach to Research in the Public Sector

A recent survey caught my eye, not for the content of the survey itself, but for what it signals about a shift in the way research is conducted in the public sphere.

As it turns out, according to this survey, people favor sick leave for public employees. You’re shocked, right? But the real story is the platform: CityFeedback (full disclosure: CityFeedback is part of SurveyAnalytics, which is run by our senior contributor, Vivek Bhaskaran.) Traditional research and feedback mechanisms, particularly in the public sector (federal, state and municipal governments) have relied on the telephone. Telephone surveys conducted on the part of the government, or telephone calls that citizens would have to place in order to lodge a complaint, make a request, and so on.

CityFeedback leverages more modern approaches to collecting data from residents, including applications for mobile devices, including the iPhone and Android platforms. One of the most significant benefits of such an approach as I see it, beyond the growing preference of electronic communication over making a phone call, is the ability to capture contextual data along with the feedback (exact location, photos, etc.) Now, when a resident wants to report a pothole, they’re report is rich with contextual information that aides the city in their response. Additionally, municipalities are able to conduct more cost-effective, direct outreach to residents, gathering their views and opinions on any number of current issues.

Have you seen this sort of technology put to use in your locale? As a resident, would you want to use tools like CityFeedback to connect with your local government? Share your thoughts in the comments, or find us on Twitter (@researchaccess).

What’s Your Beef?

Airline food for vegetarians sucks. End of story. Think I’m wrong? Talk to the hand. Seriously.

Now before you think the obvious and say “Romi, all airline food sucks so clearly vegetarian airline food does too.” Put your Venn diagrams away because my point is different.

Veggie airline food is crappy because of a Marketing and Market Research problem. Not because of bad cooks.

For those of you who have suffered uncooked potatoes or been forced to daintily pick at fruit when you are ravenous, you very likely know where I am going here.

Which is this: Too much segmentation is a BAD thing. Too much “correlation” is a BAD thing.

Because airlines seem to think that if you are vegetarian you are either 1. On a diet; 2. Hindu; 3. Lacto-Ovo-Pyscho.

In my case I’m none of the above though I probably should be 1. Being vegetarian doesn’t mean I lost my taste buds in a freak accident. Nor does it mean that I hate fat or think protein is an organization fighting for adolescents’ rights.

Bro, I am HUNGRY, just like you are.

Okay I made my point.

So just remember, figuring out “coupled traits” and “buying adjacencies” is a cool exercise but often you’ll just get nonsense data.

Remember, Marketing is more art than science.

Anyway, enough. Off to drink a Jicama shake for breakfast.

NOT.

How to Love a Loser

[Editor's Note: The  following post by Tom Ewing was originally published by and is syndicated with permission by our sister site, GameAccess.]

I have a dirty gaming secret: I’m rubbish at lots of them. Anything that involves reflexes, timing, pixel-perfect judgement or any kind of co-ordination will floor me. I am condemned by fate to be an eternal noob.

That’s not to say I don’t like games, of course. What I don’t like are leaderboards, points, hard-to-unlock levels, and badges. Just looking at those gives me the same kind of sinking feeling I had when I was a 16 year Morrissey fan thinking about going to a disco – they represent a world of achievement and fun I can only press my nose against.

That list of things I don’t like may look familiar to you – they’re also the mechanics most commonly cited when people talk about “gamification”. One of the big reasons people get excited about gamification is its possibility for engagement – introduce game mechanics and respondents (or customers) become more motivated.

Unless they’re like me, and feel less motivated.

Am I just a freak, though? (“Yes!” cries the inner Morrissey fan.) There have been plenty of studies showing the link between game mechanics and user engagement and activity. You shouldn’t design your game-like things exclusively for a grump like me. But you should keep my sad example in mind anyway, because you have to remember not to only design for winners.

For instance, let’s move a half-step away from games and think about salaries. Salaries in a company are a kind of leaderboard, after all! Except they’re not, because they’re generally kept secret, and I suspect most people would prefer it stayed that way.

But if leaderboards and levels are so motivating, why keep salaries a secret? Well, one reason is probably that we don’t think of work as a game, but another is that if you’re off the bottom of the high-score table, and have no real hope of getting onto it, you might well end up thinking “sod this” and going off and playing another game instead.

A table of salaries, in other words, represents a hierarchy, and hierarchies aren’t fun. When I was first working on online communities in a research context I got hung up on the idea of the 90-9-1 rule – the one which says that 1% of your users will be heavy contributors, 9% light contributors, and the majority will do nothing. My initial notion was that trying to change these percentages would improve the health of a community.

I now believe I was wrong: what matters isn’t the level of contribution, it’s the level of mobility between tiers. If the 90-9-1 rule is a hierarchy – if the tiers are more or less frozen – you’ve got a static, boring, predictable community. If the tiers are fluid, you’ve got a vibrant, unpredictable one where more people will contribute when they have something to offer.

I think gamification – or at least using visible, competitive game mechanics – works the same way. If you know you’ll never get onto the leaderboard, the leaderboard isn’t a great motivator. The same goes for the badges you’ll never get, the areas you’ll never unlock, and so on. This isn’t a radical perspective – it follows from standard “gamification” advice: that you should make sure every effort a participant makes at least has the potential for reward.

But that’s not all you have to do. You also have to ensure that the aggregated effect of all those rewards is a fluid system, not a hierarchy. You don’t want to end up with the kind of game that motivates only the winners. Because otherwise losers like me might simply walk away.

About Tom Ewing
Tom Ewing is a researcher and writer based in London. A regular speaker and award-winner at research industry conferences, he helped kickstart the research community’s interest in  game mechanics and their applications with a presentation at the inaugural Festival Of NewMR. He will be doing some very exciting things involving gaming and research which he’s about a week away from being able to talk about. His favorite games are the Civilisation and Pokemon franchises. Find out more about him here.

What We’re Reading: Sample Size, Mobility, Emerging Practices

Good Monday to you all. We thought we’d start out the week with a list of some of the articles and posts that have most recently caught our eye. Let’s dive right in!
  • How to Determine Sample Size for Segments – As always, Michaela Mora delivers some terrific educational content over at Relevant Insights. In this recent post, she discusses how to go about determining the most effective sample size for individual segments of an audience in order to get the most reliable results. Michaela walks us through the right questions to ask, as well as two possible approaches to take when adopting a sampling strategy.
  • Mobile Survey Response Rates ROCK – Ivana Taylor over at the QuestionPro Blog calls our attention to a recent study in which a mobile response option was added to more traditional data collection methodologies. The results were impressive: 25,000 text message responses were received from 23,500 unique mobile numbers in reply to a single open question.
  • GRIT Sneak Peek: The Top Emerging Market Research Techniques: Our friend Leonard Murphy over at the GreenBook Blog has a preview of the annual GreenBook Research Industry Trends (GRIT) study. The full report will be available in the next week or so, but in the meantime, Lenny takes a look at some of the preliminary data, with a focus on emerging market research techniques in 2011.
What are you reading this week? Be sure to share your favorite articles/blog posts/chat discussions/etc with us, here in the comments or via Twitter (@researchaccess).

The Intersection of QR Codes and Gamification

Earlier this week, we wrote about the current state of QR codes (or at least opinions in the current state), as well as the importance of contextual feedback and the proliferation of real-world hyperlinking.

Today, Betty Adamou, editor of our sister site GameAccess.com, has a terrific piece looking at the intersection of QR codes, gamification, and market research.

Betty discusses new and unique uses of QR codes, and also presents some great ideas for how game theory and QR codes can be combined to enhance approaches to market research. For example:

You then walk on to the cosmetics counter for Chanel and would love a discount. No problem, just scan the Chanel concession QR code, answer the question and get 5% off any purchase.

  • Answer 5 QR Code questions around the mall and you get a discount from say, McDonalds.
  • Answer 10 QR code questions and you can get a further 10% off any sale item.
  • Answer 20 QR codes and you get 50% off dinner for two in a choice of the mall eateries.

As Betty points out, this approach not only increases engagement for the customer/respondent, but also increases both sales and quality of research data (we now have context!)

So go check out Betty’s latest article, and let us know what you think. Leave your comments on this post, or find us on Twitter (@researchaccess).

PS: Some of you may already be familiar with my love of infographics. (It’s very real.) Well, I just came across this infographic documenting the rise of QR codes. Among the statistics conveyed in the graphic:

  • QR code uptake has increased 4589% from early 2010 to early 2011
  • 56% of QR codes appear on product packaging
  • The majority of users expect to receive a coupon or deal from scanning a QR code
  • 11 out of 50 Fortune companies are incorporating QR codes into their marketing strategy
  • 68% of QR codes are scanned via an iPhone
Take a look!

Early Demises: Why Zealotry around Digital Hurts Marketing

It is NOT all about digital. I swear I read that somewhere in a magazine. Or, maybe, I saw it on TV. Not sure, but I do know that Marketing is a many-splendored thing and shouldn’t be cast as narrowly as it has been of late.

This piece is an admonition not a statement of nullification.  But we need to make it clear that Marketing is still about the “mix” and about “context.”  So the intended audience is those of you who are in “digitally zealous” mode – head out of orifice please!

My favorite piece of Marketing over the past 5 years has to do with the TSA. Yes the Transportation Safety Administration. No, not their cool outfits.  No, not their incredible customer-service. And not the **** probes either.  Instead, I love the trays. Because Zappos “owned” them for a long time and it made sense: advertising shoes where you put your, well, shoes.

Nothing like a captive audience, already immersed in “context.”

I have to admit I still believe in magazines too. While I think there are serious considerations around the environment that might militate against the growth of the magazine base in general, I still subscribe to no less than 10 magazines. In many ways, I still have a bias towards what I call the “burden of authorship” which characterizes the print editorial process. Because of that, I pay more attention to the advertisements in magazines because I find the surrounding content valuable.

And so on. You get the point.

Look, I embrace digital too – when it makes sense. The possibilities unmasked by the digital media are fantastic: measurement, quick-revolution, interactivity, deep experiential value, and so on. Wow. Because of these, it’s easy to get sucked into that world and to think “Digital is everything.”  It’s easy to get sucked into religion as well but so far I’ve steadfastly avoided that as well.

Because zealotry of any fashion hurts the whole.

So as Marketers, let’s reintroduce ourselves to our complex and wonderful craft.

In all its forms.

[Editor's Note: This post originally appeared on our sister site, MarketingAccess.com]