Survey accuracy is the extent to which a questionnaire result represents the attribute being measured in the group of interest or population. To determine how accurate are the data captured by a questionnaire and how much reflect the entire population, requires computing the confidence interval and the confidence level.
Professional Enterprise ReportR DistributR This guide will teach you:
1. Confidence interval and confidence level
The confidence interval (also known as the “Margin of Error” or simply "Error") is usually expressed as a plus or minus percentage, e.g., “+/- 5%”, which indicates that the questionnaire means score likely deviates from the population mean for that attribute by less than 5%. For example, if 30% of the respondents pick a certain choice on the questionnaire and you have a margin of error of 5%, you can be "sure" that between 25% (30% minus 5%) and 35% (30% plus 5%) of the entire population would pick the same answer.
How sure can you be? This is what the confidence level (Also known as "Confidence") tells you. It’s listed as a percentage and tells you how certain you can be of your results.
2. Survey accuracy standards
For example, a confidence level of 95% means that if you conducted your questionnaire 100 times, you would come up with the same results 95% of the time. Most questionnaire research uses a 95% confidence level since this strikes an optimal balance between accuracy and cost. This is why we seldom hear the confidence level presented. In some circumstances, we might use a higher confidence level (say, 99%) or a lower one (say, 90%).
You should include key demographic questions in each questionnaire so you can see if the survey sample correlates to your membership. If 75% of your members are from the West, for example, responses to a general membership survey should also be 75% from the West. Some types of response biases can be statistically corrected; others may require you to conduct a second mailing to make sure the respondents accurately reflect the membership composition.
3. How can survey error be reduced?
- Use the correct sampling strategy to reach your target population. This includes correctly identifying the population you seek to send your questionnaire and making sure that your sample is representative of them. Also, make sure that the way in which you are reaching your sample is not introducing systematic bias – the phone book may not be an appropriate way to reach people if most of your population are young and most likely only have a cell phone.
- Try to reduce bias in responses. If your questionnaire includes sensitive questions, be aware that respondents may answer in a way that ends up under-reporting or over-reporting the issue, for example, bullying, domestic violence, or alcohol use. Try to frame the questionnaire and the questions in a non-threatening way so as to encourage respondents to be honest. Anonymity can help respondents feel safe to express their true opinion.
- Improving the response rate. Having a low response rate to the questionnaire can significantly affect the validity and reliability of the responses and analysis. Make the questionnaire interesting to respondents (why not try adding our scratch card or slot machine to add some fun!) and take care of the layout and presentation to make it appealing. Find other ideas here to increase response rate.
What's next?
- A double-barreled question is composed of more than two separate issues or topics, but which can only have one answer. It is also known as a compound or double-direct inquiry. These inquiries occur mostly in two very different circumstances: in research and in court.
- The sample size refers to the number of individual pieces of data collected in a questionnaire. The sample size is important in determining the accuracy and reliability of a questionnaire's findings. In practice, the sample size used is determined based on the expense of data collection, and the need to have sufficient statistical power.
- Respondent burden is a relatively recent concern and it is often defined as the effort required to answer a questionnaire, or more precisely, how the responder perceives the participation in terms of how long it will take, difficulty level, and emotional toll.