1. Item Analysis What makes a question good??? Answer options?

2. Multiple Choice Questions Elements of a good multiple choice question????

3. Item Analysis How can we determine if it is a good question????? n Distractor power n Item difficulty n Item discrimination

4. Distractor power Strength of each distractor (should be equal) # answering incorrect # of distractors

5. E.G. n There are 4 possible answers n Option a is correct n if 9 people choose option b (9/3 = 3) n and 12 people choose option c (12/3 = 4) n and 15 people choose option d (15/3 = 5) n then option d is the strongest distractor

6. Item Difficulty = p (percent of people passing) p = number answering correctly total number answering n If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question? p 1 = 34/100 =.34 n For 65 correct p would be 65/100 =.65 SO… the higher the p, the easier the question (score must be between 0 and 1)

7. Item Difficulty is... n A behavioral measure n Based on the current group n Allows internal comparison n Also allows external comparison

8. In construct based tests n Item difficulty is similar to item endorsement n Number agreeing with an item

9. Item endorsement also... n A behavioral measure n Based on the current group n Allows internal comparison n Also allows external comparison

10. Item discrimination = d Does the question discriminate between those who do well overall and those who do poorly? (or on a psychological test, those scoring high or low on the construct)

11. Distribution To determine level upper (U) = top 25 to 33 % of scores lower (L) = bottom 25 to 33 % of scores

12. Determining item discrimination d = U-L d = U-L n where d = item discrimination U = number in the upper group who get question correct L = number in the lower group who get question correct n = number in one group (U or L)

13. Examples of d n With 33 in U and L n if all in U and all in L get it correct 33 - 33 / 33 = 0 n if all in U get it correct, and none in L 33 - 0 / 33 = 1 n if none in U get it correct, and all in L 0 - 33 / 33 = -1 n if 30 in U and 10 in L get it correct 30 - 10 / 33 =.61

14. Value of d n Range for d = -1 to 1 n higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group) n negative d indicates a really bad question (-1 indicates all in lower group got it correct and none in upper group)