Item Analysis

________________________________________________________________

1. Score each answer sheet, write score total on the corner * Obviously have to do this anyway

2. Sort the pile into rank order from top to bottom score ( 1 minute, 30 seconds tops)

3. If normal class of 30 students, divide class in half

* Same number in top and bottom group;

* Toss middle paper if odd number (put aside)

4. Take “top” pile, count number of students who responded to each alternative * Fast way is simply to sort piles into “A”, “B”, “C”, “D” // or true/false or type of error you get for short answer, fill-in-the-blanks Or set up on spread sheet if you’re familiar with computers

ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B 4 C 1 D 1 O*= Keyed Answer|

* Repeat for lower group

ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B 4 2 C 1 D 1 O*= Keyed Answer|

* This is the time consuming part but not bad, can do it while watching TV, because you’re just sorting piles ________________________________________________________________ THREE POSSIBLE SHORT CUTS HERE (STEP 4)

(A) If you have a large sample of around 100 or more, you can cut down the sample you work with * Take top 27% ( 27 out of 100 ); bottom 27% ( so only dealing with 54, not all 100) * Put middle 46 aside for the moment

* Larger the sample, more accurate, but have to trade off against labour; using top 1/3 or so is probably good enough by the time you get to 100; -- 27% magic figure statisticians tell us to use * I’d use halves at 30, but you could just use a sample of top 10 and bottom 10 if you’re pressed for time * But it means a single student changes stats by 10%

* Trading off speed for accuracy…

* But I’d rather have you doing ten and ten than nothing

(B) Second short cut, if you have access to photocopier (budgets) * Photocopy answer sheets; cut off identifying info

(can’t use if handwriting is distinctive)

* Colour code high and low groups dab of marker pen color * Distribute randomly to students in your class so they don’t know whose answer sheet they have * Get them to raise their hands

* For #6, how many have “A” on blue sheet?

How many have “B”; how many “C”

* For #6, how many have “A” on red sheet…

* Some reservations because they can screw you up if they don’t take it seriously * Another version of this would be to hire kid who cuts your lawn to do the counting, provided you’ve removed all identifying information * I actually did this for a bunch of teachers at one high school in Edmonton when I was in university for pocket money

(C) Third shortcut, If you can’t use separate answer sheet, sometimes faster to type than to sort

SAMPLE OF TYPING FORMATFOR ITEM ANALYSISITEM # 1 2 3 4 5 6 7 8 9 10KEY T F T F T A D C A BSTUDENTKay T T T F F A D D A CJane T T T F T A D C A DJohn F F T F T A D C A B|

* Type name; then T or F, or A, B, C, D == all left hand on typewriter, leaving right hand free to turn pages ( from Sax) * If you have a computer program – some kicking around – will give you all stats you need, plus bunches more you don’t – automatically after this stage -------------------------------------------------

OVERHEAD: SAMPLE ITEM ANALYSIS FOR CLASS OF 30 ( PAGE #1) (IN TEXT) 5. Subtract the number of students in lower group who got question right from number of high group students who got it right * Quite possible to get a negative number

ITEM ANALYSIS FORM TEACHER CONSTRUCTED TESTSCLASS SIZE = 30ITEM UPPER LOWER DIFFERENCE D TOTAL DIFFICULTY1.A 0 *B...