Evaluation Tip 4:Recognizing mistakes in survey questions

One common way to do an evaluation is to use a survey.When using or developing a survey, it is important to be able to recognize and avoid writing bad questions. There are certain characteristics that you can look for to ensure that your questions are written properly.

Common Mistakes in Survey Questions:

1. Double-barreling

Double-barreling occurs when one question deals with two or more issues or ideas, which makes it impossible to decide which issue the response really addresses. It is important to address one issue at a time and to use one question for each issue.

2. Leading

Leading occurs when a question encourages a person to respond in a certain manner (positively or negatively). Survey questions should be written in a manner that allows the person to respond without being influenced.

3. Wrong assumptions

Making a wrong assumption occurs when a question is written based upon the idea that the person answering has prior knowledge that is necessary to respond, when they may not. The best approach to writing answerable questions is to keep them simple. When that isn�t possible you can expand them or create multiple questions when needed to ensure that each person can accurately respond.

4. Missing response options

The response options for questions must provide an option for every possibility that can accurately answer the question. When an item does not provide enough response options, you risk losing useful information about that which does not fall into the available categories.

5.Incorrect use of scales

One example of incorrect use of a scale occurs when a numerical scale with three or more response options is implemented for a question that can be better answered by only two options. Using appropriate scales will save you a considerable amount of time and effort, and your data will make more sense.

6.Negative behavior responses

If a question is written so that a person can only choose responses that incriminate them for negative behaviors, the response is seen as forced and poorly written. It is often difficult to obtain personal information from surveys, especially that which signifies some negative aspect about a person or their behavior. When seeking this type of information, it is important to phrase your question in a manner that is not intrusive.

Testing Survey Questions

The best way to know if your questions are going to get the answers you need is to test your survey before you use it to gather data. Have a person who has the same understanding of the issues being addressed as those people the survey is intended for review the survey and tell you: 1) if the questions are clear, 2) what he or she thinks each question is asking, 3) if there should be response options other than the ones you have listed, and 4) if they think valuable information will be obtained from these questions.

You can find more information about writing good questions at these links:

http://www.ryerson.ca/~mjoppe/ResearchProcess/WriteBetterQuestion.htm

For additional information contact: Launa Mallett, 412-386-6658, LMallett@CDC.GOV