Item Analysis and types of item analysis
Item Analysis allows us to observe the characteristics of a particular question (item) and can be used to ensure that questions are of an appropriate standard and select items for test inclusion. Introduction Item Analysis describes the statistical analyses which allow measurement of the effectiveness of individual test items. An understanding of the factors which govern effectiveness (and a means of measuring them) can enable us to create more effective test questions and also regulate and standardize existing tests. The term item analysis refers to a group of statistics that can be calculated for individual test items. There are a variety of item statistics that can be computed and a variety of calculation techniques. The Three most commonly used statistics are item difficulty, item discrimination and distractor power. Although these statistics are usually discussed in regard to multiple-choice ability tests, two of them, item difficulty and item discrimination, can be adapted to the analysis of short-answer and essay questions and are also used on personality, interest, and altitude tests.
Purpose of Item Analysis
Evaluates the quality of each item Rationale: the quality of items determines the quality of test (i.e., reliability & validity) May suggest ways of improving the measurement of a test Can help with understanding why certain tests predict some criteria but not others
When analyzing the test items, we have several questions about the performance of each item. Some of these questions include: Are the items congruent with the test objectives? Are the items valid? Do they measure what they're supposed to measure? Are the items reliable? Do they measure consistently? How long does it take an examinee to complete each item? What items are most difficult to answer correctly? What items are easy? Raumish Masud Khan Assistant Professor Department of Applied Psychology Kinnaird College For Women Lahore
Psychometrics I (PSY - 401)
Are there any poor performing items that need to be discarded?
Types of item analysis
There are three common types of item analysis Difficulty Index Discrimination Index Analysis of Response Options
Difficulty Index – (Assess difficulty of the items)
Teachers produce a difficulty index for a test item by calculating the proportion of students in class who got an item correct. (The name of this index is counter-intuitive, as one actually gets a measure of how easy the item is, not the difficulty of the item.) The larger the proportion, the more students who have learned the content measured by the item. Item-difficulty analysis is appropriate for maximal performance tests— achievement and aptitude tests—because the analysis requires that test items be score as correct or incorrect.' It is not appropriate for typical performance tests, such as personality tests or interest inventories. The most common measure of item difficulty is the percentage of test takers who answer the item correctly. Referred to as the item-difficulty index, it is represented by the symbol p and calculated as follows: p = Number of persons answering item correctly
N In which p = item difficulty for a particular test item N = the total number of people taking the test
General Rules of Item Difficulty… Raumish Masud Khan Assistant Professor Department of Applied Psychology Kinnaird College For Women Lahore
Psychometrics I (PSY - 401)
p low (< .20) p moderate (.20 - .80) p high (> .80) difficult test item moderately diff. easy item
Discrimination Index – ( Assess how well an item differentiates between high and low performers) The discrimination index is a basic measure of the validity of an item. It is a measure of an item's ability to discriminate between those who scored high on the total test and those who scored low. Though there are several steps in its calculation, once computed, this index can be interpreted...