In a prior post I highlighted an interesting statistic – for any given project market researchers can now expect up to 40% response on smartphones. This percentage will continue to increase. The question then becomes one of what do we know about mobile responders and can this be applied to our sampling plans?
First and foremost, mobile responders are less likely to actually complete the surveys they start. Depending on the source cited mobile respondents are between 2.6 and 4.0 times MORE likely to drop out of a survey before completion. Designing for mobile optimization can help reduce this rate, but it is wise to over sample mobile respondents if you want a balanced sample. Consider oversampling by a rate of 3:1 for mobile respondents compared to desktop or tablet respondents.
Speeding is another behavior to watch with mobile respondents. There is conflicting evidence regarding completion time for mobile respondents. Some sources report that mobile respondents will complete their surveys in less time than desktop or tablet respondents. Other authors indicate the opposite. The question then becomes does a faster response time mean lesser quality data? If you are asking for gut reactions or attitudes on familiar topics then speed is fine. However, in other cases speed can easily equate to not reading instructions. It may be unreasonable to expect mobile respondents to complete the same discrete choice exercise as their desktop counterparts.
In regard to discrete choice, it is possible to lighten the load by reducing the number of tasks, but exposing the experiment to a larger set of respondents. This would make for a more effective approach if you are allowing mobile respondents into the survey.
A final point of consideration is the need to begin using response device as a variable for screening data quality. Would you assume differences between mobile and non-mobile respondents makes the mobile respondents of lesser quality? As analysts we need to ascertain if mobile respondents are different because they are mobile or if they are from a different demographic. Most, if not all, online survey platforms will export device as a variable. Are you using it in your data quality assessments? If not, then you should be!