You Probably Don’t Think You’re Shallow

October 5, 2010 By Leave a Comment

A research manager working in a corporate market research department recently told me, “We have to re-write all the reports delivered to us by all our market research vendors because the data is too voluminous and the recommendations are too shallow.” The manager didn’t have one particular research supplier in mind.  Instead she made a blanket statement about all suppliers.  She didn’t sound particularly upset about it.  She simply resigned to herself that shallow recommendations are the way it is and always will be.

Having been on the receiving end of the supplier recommendations, I have had the same experience while being part of a corporate market research group.   But I’ve also spent plenty of time on the delivery end of supplier recommendations as part of research supplier agencies.  And I don’t share her resignation to the fact that it always has to be that way.

How recommendations often work (or don’t work.)

A few years back a research supplier of mine (while I was on the corporate side) made the recommendation that my company (a wireless phone carrier) should explore the opportunities for offering data services like text messaging, in a way that is easy to use.  They went on to support the recommendation by presenting research findings that showed how consumers will increasingly use wireless networks for services beyond voice calling.    The research supplier certainly wasn’t wrong about that.  That observation now seems obvious in retrospect.    But at the time, when wireless phone users were much less concerned about data services, like text messaging, that finding was the type of thing research agencies hung their hat on.  That’s how they hoped to add value.

The challenge then (and now) is how to take a general finding such as increased usage of data services and turn that into and actionable marketing and product development strategy.    When my company’s product group was presented with the recommendation of easier to use data services, it was met with a polite but sarcastic response that could be summed up as “no – duh.”  They had to bite their lip to prevent saying things like “Is that what we pay for when we fund the market research budget?”

It doesn’t have to be that way

A market researcher needs to stand with one foot outside the market research function.  Have conversations with various decision makers in the organization.  Talk with people who don’t have any words like market research in their job title.   Learn what their decision making environment is like.  This means learning about decisions made in the past, specific goals for the company, external influences, and as much relevant info as possible on the kinds of things decision makers really are concerned about.   Gaining just a bit of experience outside the market research function goes a long way toward having perspective and context that may be very useful when writing recommendations.  It may also prevent your findings and recommendations from being met with a big “so what.”

Suppose the market researcher who thought they’d earn a gold star by delivering the finding about increased data services for cell phone users had done more.  Suppose they went a bit further to talk to more people at their client company.  They may have had a conversation with the manager who oversees everything related to handset functionality.  They may have learned that the using a standard handset key pad with only numbers and a few other buttons was a cumbersome and limiting design feature.  The researcher would have heard things like “People hate using the 3 touch number pad to type in letters.    And the predictive text, T9 typing mode sometimes takes more time to correct than it’s worth.”  From there, tests on the idea of a QWERTY keyboard on a handset could have been designed.  This could lead to research findings that have a real chance for the client to make decisions based on research.  Instead of a yawn and a comment like “ya, thanks, I already know that,” the client will have some real useful learning and maybe even feel a bit inspired by research results.

Walk the walk if you call yourself a strategic partner

The cost of conducting market research with an agency that calls itself a full service research and consulting firm can be 2 or even 3 times higher than the cost of conducting research with an agency that more modestly features their data gathering and reporting capabilities.   But each type of agency can be equally capable of performing the core market research services.  Each type of agency often can adhere to all the rigorous requirements of collecting data free of bias or other problems.  So why do some clients pay many thousands more for a “full service” research and consulting firm.   The answer presumably is that the full service firm delivers consulting and recommendations.   But this can be a bit like a job applicant that bluffs his way through a job interview by talking conceptually and not really providing detailed and specific information.    The way a full service agency with full service fees earns all that extra money is in delivering recommendations that have depth, implications, specificity, and perhaps even become actionable.

Be like Sherlock Holmes

A piece of advice that an early mentor of mine passed on to me is to “be like Sherlock Holmes.”  It took me quite some time before I truly figured out what he really meant by that.  What skills did Sherlock Holmes have exactly?   He was endlessly curious and forever investigating – looking for clues.   A researcher similarly has to be on the lookout at all times for information to piece together.  It’s like a game of connect-the-dots so that the whole picture can be seen.  A challenge is that an external market research partner often does not have access to all the dots.  Some of the dots are hidden in the client’s customer database.  Some of the dots are hidden in past research.  Some of them may not even be dots until they are combined with insights from people outside the market research function.   Wherever the clues come from, it is the researcher’s job to figure out how to tell a story based on connecting the dots.

In the spirit of Open Access

So just as the title of this online forum calls for, a research supplier must maximize all chances they have to get access to information and other relevant research findings.   And the gatekeepers of the information must be made to share access in a usable way that is as targeted as possible.  When the pieces are put together, stones are not left unturned, and learning is gathered from plenty of sources, then maybe, just maybe, a set of recommendations can be made that won’t be met with an accusation of shallowness.

Filed Under: Market Research Tagged With: , , , ,

Don’t say “Sorry, we didn’t ask that,” instead let the people shine through

July 10, 2010 By 1 Comment

It’s among the most uncomfortable moments a market researcher can have.  You’re standing in front of clients presenting the results of a study.  All eyes are fixed on you.  They’re listening to your every word.  And an unexpected question from someone in the audience gets put to you on what the study has learned about a particular topic.   But the study didn’t ask about that topic.  It wasn’t included in the questionnaire.  You may be tempted to take the easy way out and put the responsibility back on your client and remind them they had a chance to raise that issue back when the study was being designed.  That would be a way to get out from underneath the unexpected question that many researchers would take.   But there’s another answer that will make you look like a star, and not somebody who squirms away from unexpected questions.

And that answer goes something like this: “We listened to the comments of 400 people when they were asked their thoughts on three different areas very much related to your question.  They had plenty of opportunities to raise that same issue (or concern) a number of times and it was mentioned by just two people.  And here’s exactly what those two people had to say…”    This answer gets the client some good feedback on their issue and it prevents you from looking like a researcher who missed the mark on something.

The way to be prepared to give this answer does not require methodological genius.  It requires writing some open end questions that are just specific enough to stay on target, but not so specific that a respondent doesn’t get to say what’s on their mind.  It also requires spending a few hours going through all the open end responses.    Yes, it requires getting into the nuts and bolts, into the weeds, into the details.  But before you claim your time is too valuable for such a chore, consider that spending time with verbatim comments will prepare you for a presentation in ways beyond how the numbers can prepare you.

Verbatims do even more

You’ll find that after you are armed with many particularly insightful verbatim comments that your presentation of the data will take on far greater impact as well.  It is very good advice to say that the way to present data is to tell a story about what it all means.  Nobody wants to be subjected to page after page of just numbers.  And your story about what it all means will be a much better story if it is peppered with timely and relevant real comments from real people.  You’ll have much more confidence that you really know what respondents have to say about the study’s objectives and what they have to say about plenty of other things too.  In short, you’ll be much better prepared to deliver an outstanding presentation.

And as far as a deliverable to a client, I’m not talking about a collection of just a couple dozen comments.    It should be a thick stack of many hundreds (perhaps thousands) of comments from a quantitative study.  But you can’t just present a big collection of random comments.   They need to be organized by category and quantified.   That’s right – quantified.  How many people said this type of comment, and how many people said that.  About 50 different categories of comments is typical.   Focus groups can’t do that reliably enough to make statistically sound projections to a larger population.   And if you do the coding and classifying yourself you’ll be left with a supplemental file to your presentation deck that is much more valuable than a few pages in your stack of data tabulations that say “open ends.”  After all, those tables reduce the rich and often colorful verbatim comments to a listing of categories just a few words long.  Those tables most definitely do not provide the same perspective as getting into actual respondent words to hear for yourself.

Watch how they are drawn to it

When preparing for a client presentation you probably have painstakingly built slides that expertly display your statistical prowess.  Many researchers like to show off conjoint analysis and regression models.  It makes all that time studying statistics pay off.   (I’m no different.  I like to take a client through cluster analysis.  And I love to hear questions like “why did you decide to do discrete choice analysis?”)  But if you look around the room after a presentation is over, you may notice something interesting.  Among the back-up materials that were created to support the main presentation deck, it is the collection of actual respondent comments that everybody wants to pore through.   After all, this is what real people have to say in real words.  It is a level of communication that many people need to experience before they can be truly persuaded.   You’ll see how one corporate VP will peruse the verbatims for just a minute or two and then nudge another corporate VP and say ‘listen to this one.’   I’ve even known of a client who didn’t bother to keep and file his copy of the summary report, but he was careful to keep his copy of the verbatim comments from customers.   He wanted to feel a closer connection to his company’s customers.

So the client who likes focus groups because he/she needs to hear it from the horse’s mouth will be satisfied and the client who is only convinced by quantitative proof will also be satisfied.   So a quant study has an element of qualitative findings, and those qualitative findings have an element of quantitative analysis.  The goal is to make the presentation and analysis come alive and deliver the best of what both types of research can deliver.

Filed Under: Market Research