When you analyze and report survey results, focus on the story you want to tell – don’t assume the questionnaire’s structure is the best for analysis.
A properly designed questionnaire provides a narrative thread that conversationally moves the respondent from one question to—if possible—another closely related question, abruptly changing topics as few times as necessary. The order of specific questions might be further dictated by the need to screen in (and screen out) respondents with particular demographics, to collect verbatim responses up front before biasing perceptions with subsequent closed-ended questions, and to wrap up by collecting important but personally intrusive demographic information. ￼￼
￼While this questionnaire structure may come to seem inevitable, especially after multiple rounds of revision, it was designed to improve the respondent experience. It may, however, have little or no relationship to the proper order of material for analyzing and presenting the results.
Tip 1 – Your job is to rearrange the questions into an order that best tells the story of the data.
For great reports, take a tip from great reporters – follow the inverted pyramid style of reporting: answer up front the essential business questions that prompted the research in the first place. Start with your conclusions and support them, rather than accumulating evidence. Only then give the detail for those of your readers with the time and interest.
Think of the interest levels of 3 types of readers as you write your report:
- Type 1 wants the gist of the study;
- Type 2 wants the conclusions with supporting details;
- Type 3 wants the details of the entire body of research.
You want each of these types to be able to take something from your research report that is relevant to their work. Whether senior management or junior analyst, there will be something for each of them in your report.
￼￼Write the report as if the reader won’t read all of it! Journalists in the 19th century often wrote flamboyant, meandering news accounts that took their time to get to the actual story that they were reporting. The growing professionalization of journalism 110 years ago stimulated the development of the inverted pyramid style of reporting, with a summary news lead followed up by further detail. The advantage? Editors could cut the article to fit a specific physical location in the newspaper while still reporting the most important details. The added benefit for readers was that they could skim headlines and first paragraphs to get a general sense of the news and could stop at any point. Create your report in the same way!
Tip 2 – Don’t assume the existing question choices are the best way to present the results either.
By adding new variables you can present the results a more presentation-friendly manner. Most survey systems simply spit out question results in questionnaire order, and show the actual choices given. Add-on analytical packages, on the other hand, provide users the flexibility to restructure the data to better discover insights.
For example, i a recent survey, four target markets were studied and respondents were screened in or out based on their Metropolitan Statistical Area. Yet whether the respondent was in Savannah or Kansas City, Charleston or Oklahoma City was irrelevant; what was important was that a radio campaign was being tested in two of the cities (the Test Markets) and not the other two (the Control Markets). Accordingly, a “Market” variable was created that encoded this relationship. The market in which the respondent was most important, with the city only being indirectly of interest. Additionally, a variety of national campaigns, including “Social media”, “Web search”, and “Web banner ads” were tracked. The business decision makers were keenly interested in the success of web marketing overall, these choices were grouped for presenting certain analyses, resulting in a much better presentation of the findings.
The questionnaire itself represents only some of the building blocks for your analysis; don’t treat it as the framework or blueprint. Depart from it in terms of structure and data points.
Want more tips to produce analyses that will drive your success? The above is an excerpt from a new white paper I wrote for MarketSight. I encourage you to download it: 10 Keys to Survey Analysis.
Jeffrey Henning, PRC, is president of Researchscape International, which provides “Do It For You” custom surveys at Do It Yourself prices. He is a Director at Large on the Marketing Research Association’s Board of Directors. You can follow him on Twitter @jhenning.