Sometimes in the day-to-day work of conducting and interpreting market research, it’s easy to forget that many people who work with surveys on a daily basis have not had formal training in statistics. Even for those who have been trained, it can be useful to have a refresher from time to time.
UNDERSTANDING MARGIN OF ERROR
One of the most basic concepts in market research is the confidence interval, commonly referred to as the “margin of error.” The confidence interval is a range of values within which a survey result can be assumed to accurately represent the underlying construct being measured.
Technically the margin of error is half the confidence interval; plus or minus 5 percentage points represents a confidence interval of 10 percentage points
The general public has a basic if vague understanding of this concept. Indeed, media reports of election surveys often report a result “plus or minus” a certain number of percentage points.
The confidence interval is important because it helps us as marketers and researchers understand the limitations of our survey results. The confidence interval estimates the inaccuracy of our results due to “sampling error,” that is, error stemming from the limitation of conducting our survey among a single sample of the population of interest (rather than the impractical or impossible alternative of conducting a census of the entire population).
Sampling error is distinct from other types of survey error – including measurement error, coverage error, and non-response error – but those are topics for another time.
Here are the factors that affect the margin of error:
- confidence level
- proportion in the sample
- sample size
Confidence level. You must choose how statistically certain you want to be. The most common confidence level is 95%. The conceptual meaning of a 95% confidence level is as follows. If you were to conduct your survey one hundred times with randomly drawn samples and everything else were equal, the result of your survey question would be expected to fall within the confidence interval ninety-five of those times and outside it five times.
Proportion in the sample. Proportional estimates closer to 50% are subject to more variability than estimates near the ends of the spectrum, e.g. 10% or 90%.
Sample size. The greater the sample size, the lower the margin of error because variability due to sampling anomaly is reduced.
CALCULATING MARGIN OF ERROR
There are three ways to calculate the margin of error: use a formula, use a look-up table, or use an online calculator.
Use a formula. There are a number of formulae you can use with slightly varying assumptions. If you want to go through the calculations yourself using a formula, I refer you to this web page: “Guide to Computing Margins of Error for Percentages and Means” from Professor Ted Goertzel’s at Rutgers University, who explains the calculations better than I can hope to do.
Use a look-up table. Here’s a table that will be appropriate in most circumstances. This table is based on a 95% confidence level. In order to find the confidence interval (the “plus or minus” amount) for a particular proportion, go the the row closest to the proportion of interest and the column closest to the sample size of interest. For example, if an N=500 election poll showed a race tied at 50% to 50%, you would go to the 50% row and the N=500 column, yielding a margin of error of plus or minus five percentage points.
| N | N | N | N | N | |
| Proportion | 1,000 | 750 | 500 | 250 | 100 |
| 10% | 2% | 2% | 3% | 4% | 6% |
| 20% | 3% | 3% | 4% | 5% | 9% |
| 30% | 3% | 4% | 4% | 6% | 10% |
| 40% | 3% | 4% | 5% | 7% | 10% |
| 50% | 3% | 4% | 5% | 7% | 11% |
| 60% | 3% | 4% | 5% | 7% | 10% |
| 70% | 3% | 4% | 4% | 6% | 10% |
| 80% | 3% | 3% | 4% | 5% | 9% |
| 90% | 2% | 3% | 3% | 4% | 6% |
Use an online calculator. The above exercises are great, but guess what, you’re in luck! There are many online calculators out there. Here are two examples:
American Research Group
Relevant Insights
I hope this post is useful as you navigate the world of survey research. Good luck, and happy polling!
About Dana Stanley
