If the greatest challenge of conducting market research is getting a qualified sample to respond to a survey instrument, the second greatest challenge has to be getting them to complete that instrument. In almost any survey any of us have ever conducted, we’ve seen the dreaded “Number abandoned” statistic, describing the number of people who began our study, but for whatever reason, didn’t complete it.
Fortunately, there’s some good information out there to help us understand why this happens!
This week, Jeffrey Henning has a post on his blog, Voice of Vovici (a regular must-read), discussing this very topic. Jeffrey refers to a study presented at this year’s Online Research Methods conference by James Sallows, vice president of EMEA Client Operations for Lightspeed Research. Definitely check out Jeffrey’s post for the full detail, but here’s the gist:
In one analysis that Lightspeed conducted, 35% of incompletions were because of the subject matter of the survey, 20% were due to media downloads, 20% due to survey length, 15% due to grids and 5% due to too many open-ended questions. Here’s what James had to say about each in turn:
- Subject matter – “Fabric conditioner is a dull subject matter, and it was an issue [for incompletes] early on in the questionnaire.” Automotive surveys, on the other hand, have a lower rate of incompletion compared to most other subjects; respondents are engaged with the topic.
- Media downloads – Older respondents won’t wait for long download times; 18- to 34-year old respondents, on the other hand, are more willing to wait while media is downloading.
- Survey length – Many respondents abandon the survey at the introduction, then remain, with a steady increase in incompletes after 15 minutes.
- Grids – One client frantically called James into a meeting due to problems with “bad” respondents. They said, “Something has to be done! Look at the data – respondents are flatlining after the 35th grid!” Even relatively small grids seem like a lot of work to respondents, let alone grid after grid. “Making grids mandatory pushes people to drop out – there is a selection of people that don’t want to do them.”
- Open-ends – Having too many questions that require respondents to type in answers leads to greater incompletion.
If you have ever conducted or will conduct a survey in your career (and hopefully, if you’re reading this blog, you have and will!), you’ve got to read up on this stuff. It’s painful to have crossed the hurdle of engaging a respondent, just to lose them over a poorly structured survey or badly worded question. Time is money, and keeping respondents engaged will save you lots of both!