ACTIVITY 4: DISCUSS ISSUES RELATED TO THE RELIABILITY OF A PSYCHOLOGICAL MEASURE Part of long question: The activities dealing with the basic principles of psychometric theory. 6. Stage 6: Evaluating psychometric properties and establishing norms 6.1 Establishing Reliability Scheme: 1. What is meant by a person’s observed score? True score and error variance in a test score? 2. The typical sources of true variance and error variance in a test score. 3. The interpretation of a reliability coefficient and how large this value should be in order to be acceptable. 4. The interpretation of the reliability coefficient in terms of statistical significance of correlation coefficients. The difference between a high and a significant correlation. 5. Different types of reliability
a. Internal consistency- split-half reliability, Kuder Richardson 20 and Cronbach’s Alpha b. Stability: I. Test retest stability ii. Alternate form reliability. 6. Techniques for assessing the stability of a test and advantages and limitations of each. 7. Techniques for assessing the internal consistency of a test, advantages and limitations of each. 8. The importance of the standard error of measurement.
Introduction: Reliability is linked to how consistently an assessment measure measures what it is supposed to measure. (e.g. driving 120km, day 1, day 2 etc. Reliability: Car= 120km, not one day 100, next day 120km) ( Konsekwentheid) The following definition of reliability: o “The reliability of a measure refers to the consistency with which it measures whatever it measures”. o Generally, if a measure is reliable, it produces similar results when administered on repeated occasions. (Verskillende geleenthede dieselfde resultate) Consistency always implies a certain error in measurement due to unsystematic or chance factors present such as emotional state of mind, noise, fatigue and so forth. X (observed score) = T (true score) + E (error score) o R = the ratio of true score variance to observed score variance. R=verhouding van ware score teenoor waargeneemde score verskil Nelda Opperman Reliability, Variability, establish norms PYC4807 An individual’s total score is comprised of true variation and error variation. Totale score saamgestel uit ware variasie en fout afwyking o Error variation consists of random error (noise) and systematic error (bias). o True variation is indicative of the real differences in the construct that the test is set to measure. Random error can be a result of an individual’s emotional state, the temperature, tiredness, disturbances during the test and so on. Systematic error is a result of various biases that influence the result in a certain direction consistently. EXTRA INFORMATION
With regard to psychological constructs the true score is never known, reliability is estimated; there is no such thing as true reliability. Test developers make use of observed data that is used numerically to compute a reliability coefficient, which is essentially a correlation coefficient. A correlation coefficient indicates “the degree of relationship between two sets of scores. In a perfect positive correlation each individual’s score occupies the same relative position in a bivariate distribution as the other test taker’s scores. In a perfect negative correlation an individual scores the best on one variable but the worst on another variable. You are able to predict a test taker’s performance on variable two by their performance on variable one. A correlation of zero is indicative of there being absolutely no relationship between two variables, reliable predictions can thus not be made. Reliability correlation coefficients range from 0.00 to 1.00. Whether or not a correlation coefficient is significantly greater than zero is the important question to ask. These significance levels point to the amount of chance or risk of error we are willing to take in drawing conclusions from the scores we have obtained. What...
Please join StudyMode to read the full document