Bad Survey Questions
EXAMPLES OF BAD QUESTIONS & SUGGESTIONS OF HOW TO FIX THEM!
Double Barreled question: Please indicate how much you agree or disagree with each of the following statements about the childcare program. Incorrect: I feel welcomed by staff and other youth at the center Correction: I feel welcomed by staff at the center I feel welcomed by other youth at the center Biased/leading question Incorrect: Community organizing is hard. Do leadership trainings help you feel prepared for community organizing? Much more prepared Slightly more prepared Somewhat more prepared Not more prepared
Correct: The leadership trainings prepare me for community organizing. Strongly agree Double Negative Incorrect Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? (22%) Very possible Impossible Correct Do you doubt that the Holocaust actually happened or not? (9%) Very possible Impossible Possible Very impossible Possible Very impossible Agree Disagree Strongly disagree
EVALUATING SURVEYS
The following are additional examples of bad survey questions. As a group, apply our class discussion, evaluating each question and its response choices to identify one error in each question (ie. double barreled). Explain the problem that your group identifies in each question and propose an alternative that solves the problem. 1. More people have attended the movie, Gone with the Wind, than any other motion picture produced this century. Have you seen this movie? Yes No
2. In your opinion, how would you rate the speed and accuracy of your work? Excellent Good Fair Poor
3. Did you first hear about the bombing: ______ from a friend or relative ______ from your spouse ______ from a newspaper ______ at work ______ from the television or radio or other electronic media
4. How do you feel about the following statement? We should not reduce military spending. Strongly agree Agree Disagree Strongly disagree
5. People grow up in all
Double Barreled question: Please indicate how much you agree or disagree with each of the following statements about the childcare program. Incorrect: I feel welcomed by staff and other youth at the center Correction: I feel welcomed by staff at the center I feel welcomed by other youth at the center Biased/leading question Incorrect: Community organizing is hard. Do leadership trainings help you feel prepared for community organizing? Much more prepared Slightly more prepared Somewhat more prepared Not more prepared
Correct: The leadership trainings prepare me for community organizing. Strongly agree Double Negative Incorrect Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? (22%) Very possible Impossible Correct Do you doubt that the Holocaust actually happened or not? (9%) Very possible Impossible Possible Very impossible Possible Very impossible Agree Disagree Strongly disagree
EVALUATING SURVEYS
The following are additional examples of bad survey questions. As a group, apply our class discussion, evaluating each question and its response choices to identify one error in each question (ie. double barreled). Explain the problem that your group identifies in each question and propose an alternative that solves the problem. 1. More people have attended the movie, Gone with the Wind, than any other motion picture produced this century. Have you seen this movie? Yes No
2. In your opinion, how would you rate the speed and accuracy of your work? Excellent Good Fair Poor
3. Did you first hear about the bombing: ______ from a friend or relative ______ from your spouse ______ from a newspaper ______ at work ______ from the television or radio or other electronic media
4. How do you feel about the following statement? We should not reduce military spending. Strongly agree Agree Disagree Strongly disagree
5. People grow up in all