Inter-Rater Reliability Report

Variations in raters’ judgments can happen either across raters, known as inter-rater reliability or in the consistency of one single rater, called intra-rater reliability. They have a variety of aspects which can influence the judgment of a scorer, and two raters might give different conclusions about similar performances. Besides the more obvious reasons for controversy, like differences in experience or lack of agreed-upon scoring routines, it has been reported that things like teachers’ attitudes regarding students’ ethnicity, as well as the content, may also influence the rating of students’ work (Davidson, Howell, & Hoekema, 2000).