O nline Course Evaluation
L iterature Review and Findings
P repared by:
J essica Wode
J onathan Keiser
A cademic Affairs
C olumbia College Chicago
S pring 2011
Summary of Scholarly Research on Student Course Evaluations...............................................2 Recommendations for Improving Response Rates......................................................................6 Additional Reading on Online Evaluations: Annotated Bibliography...........................................7
S CHOLARLY R ESEARCH
S TUDENT C OURSE E VALUATIONS
T he validity and reliability of course evaluations
• Researchers generally consider student evaluations of instructors to be highly reliable and at least moderately valid.1,2,3
• Other methods of evaluation (such as evaluations by colleagues or trained observers) have not been found to be reliable and therefore not valid.1 • Student ratings of instructors have been found to be related to ratings of instructor’s skills in course organization, rapport with students, and fair grading; variance in organizational skill (having an organized course plan and clearly identifying what students need to do) explained most variance in student evaluations.4 • Alumni rarely change their opinions of former teachers. 1,3 • When instructors collect mid-term feedback from students and have an honest discussion about it with someone, it leads to higher evaluations at the end of the semester as well as higher final exam scores, providing evidence that good evaluations can lead to better teaching.5
• Although grades do have some effect on how students rate instructors,6 its effect is fairly low7 and can be statistically adjusted for.8 Grades do not have as large of an effect as do how much students feel they’ve learned,9 how much they felt stimulated by the class,10 and whether the class was appropriately difficult (courses are rated lower for being too easy or too difficult).11
• Contrary to the “retaliation” theory, students who do poorly in a class are equally or less likely than those who do well to complete course evaluations.12 1 Centra, John. 1993. Reflective faculty evaluation: enhancing teaching and determining faculty effectiveness (Jossey Bass higher and adult education series). San Francisco: Jossey-Bass Inc Pub. 2 Hobson, Suzanne M., and Donna M. Talbot. "Understanding Student Evaluations: What All Faculty Should Know." College Teaching 49, no. 1 (2001): 26-31.
3 Aleamoni, Lawrence M. "Student Rating Myths Versus Research Facts from 1924 to 1998." Journal of Personnel Evaluation in Education 13, no. 2 (1999): 153-166. 4 Jirovec, Ronald L., Chathapuram S. Ramanathan, and Ann Rosegrant-Alvarez. "Course Evaluations: What are Social Work Students Telling Us About Teaching Effectiveness?" Journal of Social Work Education 34, no. 2 (1998): 229-236.
5 Overall, J.U., and Herbert W. Marsh. "Midterm Feedback from Students: Its Relationship to Instructional Improvement and Students' Cognitive and Affective Outcomes." Journal of Educational Psychology 71, no. 6 (1979): 856-865.
6 Johnson, Valen E. "Teacher Course Evaluations and Student Grades: An Academic Tango." Chance 15, no. 3 (2002): 9-16.
7 Gigliotti, Richard J., and Foster S. Buchtel. "Attributional Bias and Course Evaluations." Journal of Educational Psychology 82, no. 2 (1990): 341-351.
8 Greenwald, Anthony G., and Gerald M. Gillmore. "Grading leniency is a removable contaminant of student ratings." American Psychologist 52, no. 11 (1997): 1209-1217. 9 Bard, John S. "Perceived learning in relation to student evaluation of university instruction." Journal of Educational Psychology 79, no. 1 (1987): 90-91.
10 Remedios, Richard, and David A. Lieberman. "I liked your course because you taught me well: the influence of grades, workload, expectations and goals on students' evaluations of teaching."...