Topics: Psychometrics, Assessment, Test Pages: 7 (903 words) Published: December 3, 2013
Ed5-edu: Educational Measurement and Evaluation
Instructor: Stephen Jay Co

 Common

threats to reliability

 Conventional methods for


reliability of test scores
 Techniques for

improving reliability

 Reliability

is the degree to which a test
measures something consistently.

 If

any assessment lacks reliability, it lacks

 Anything causes inconsistency within

measure is a potential threat to its


Common threats to reliability
1.) Inconsistencies between earlier and
later measures
 If students are administered tests before
and after instruction, their performance
normally changes.
 Example as students in third grade given
an aptitude tests and again same test in
their sixth grade. Inconsistent result in
this case is desirable.

Common threats to reliability
2.) Inconsistencies between test items that
supposedly measure the same skill
 Typically, a teacher will ask more than one
test item to measure a skill being assessed in
a written test.
 Inconsistency can come from;
◦ Guessing
◦ Vague questions
◦ Items designed for the same skill incorrectly
measures a different skill

This inconsistencies are undesirable and
threatens reliability of scores on these tests.

Common threats to reliability
3.) Inconsistencies between alternate skills in the
same content domain
 The complexities of skills that are taught in the
classroom create another source of inconsistency
within student assessments.
 Example , assessing student ability to deliver a
persuasive speech. inconsistencies such as topic
for speech, instructional goal of the teacher or
confidence and attitude of the student.
 Samples for assessment are not absolute. Our
goal is to take steps in the development of
student assessment to maximize the reliability.

Common threats to reliability
4.) Inconsistencies from measuring
unrelated qualities within one test
 Inconsistency occurs when a teacher uses
a single score to report student
performance on a multiple unrelated
 Example, using one score on an essay test
to indicate both the correctness of
answers to questions and the correctness
of spelling.

Common threats to reliability
5.) Inconsistencies between different rates
in the scoring of student responses
 Although not practical, it is beneficial to
have more than one rater read students’
papers, review portfolios or observe
performance in class.
 Since the use of multiple rates is
impractical, we emphasize on the
reduction of inconsistencies in one
teachers’ ratings.

Common threats to reliability
6.) Inconsistencies in decision based on
student performance
 Misclassifications of students relative to a
passing score are the results of
inconsistencies in test scores.
 Misclassification more commonly occur
among students whose true performance
is close to the passing score.

Conventional methods for
estimating reliability of test scores
1.) Test-Retest method
 A student who originally obtains high
scores should continue to achieve high
score if a test is re-administered.
 It is appropriate for tests expected to give
consistent results over time.
 Examples are intelligence tests and
vocational interest inventory.

Conventional methods for
estimating reliability of test scores
2.) Alternate-Form Method
 Involves administering two forms or
versions of a test to the same students.
 It is appropriate to use the alternate-form
method whenever two or more forms of
test are developed and used
 This verifies that the alternative test
forms are measuring the same thing.

Conventional methods for
estimating reliability of test scores
3.) Methods of Internal-Consistency

Split-half method –


Kuder-Richardson method –


Cronbach Coefficient Alpha – involves non-

Continue Reading

Please join StudyMode to read the full document

Become a StudyMode Member

Sign Up - It's Free