1. Turku 2009 – F. Kaftandjieva

2. The 6th Annual EALTA Conference, Turku 2009 – F. Kaftandjieva: “Flaw & Order: World 1 and World 2 of Item Difficulty – Synergies and Tensions”

3. Turku 2009 – F. Kaftandjieva World 1 Observed Item Difficulty World 2 Perceived Item Difficulty World 3 Misplacement Index (MPI)

4. Turku 2009 – F. Kaftandjieva

5. ‘At what CEFR level can a test taker already answer the following item correctly?’ Magic Clouds: Intra-judge Consistency A1 B1 A2 C2 C1 A2 B2B1

6. Turku 2009 – F. Kaftandjieva 2 4 5 6 7 8 9 1 3 Snork Maiden’s Art Work: Judgment Task

7. Turku 2009 – F. Kaftandjieva 9 4 6 2 8 5 7 1 3 52/552/5 Snork’s wish: Misplacement Index (MPI) 65/665/6 65/665/6 65/665/6 55/555/5 55/555/5 76/776/7 76/776/7 53/553/5 Observed Item Difficulty Difficult > Difficult Easy Easy < A machine for finding things out: a machine that calculates whether things are right or wrong, good or bad. C C+D 0.400.83 1.00 0.86 0.60 MPI 0.81

8. Turku 2009 – F. Kaftandjieva The Muskrat’s Philosophical Question: How Good Is Good Enough? MPI = 0.50 MPI > 0.65 P-R-E > 50% MPI > 0.75

9. Turku 2009 – F. Kaftandjieva The Hemulen Who Loved MPI Indices: The Collection Languages: En –232 Ge – 22 Sw – 92 Fi – 35 Skills: R –187 L – 77 W – 7 V – 29 G – 28 GV – 53 0.00.10.20.30.40.50.60.70.80.91.0 MPI judge Mean = 0.71 SD = 0.10 n = 381 Judges = 187 Sub-tests = 30 Items = 2089 76% 39% 0.65 0.75

10. Turku 2009 – F. Kaftandjieva The Hemulen Who Loved MPI Indices: Skill Comparison R G&V n = 77 L R n = 53

11. Turku 2009 – F. Kaftandjieva The Hemulen Who Loved MPI Indices: Group Comparison (Reading) n = 187 6 < n gr < 34

12. Turku 2009 – F. Kaftandjieva The Hemulen Who Loved MPI Indices: Method Comparison Test Centered 83 J – 41 I 0.79 Examinee Centered 9 J – 331 E 0.87 Test Centered 15 J – 234 I 0.71 Examinee Centered 6 J – 151 E 0.88

13. Turku 2009 – F. Kaftandjieva The Island of the Hattifatteners: Item Specific MPI Indices 0.00.1 0.20.30.40.50.60.70.80.91.0 MPI item n = 25431 Mean = 0.72 SD = 0.21 Items = 2089 Tests = 30 Judges = 187 71%

14. Turku 2009 – F. Kaftandjieva The Secret of the Hattifatteners 0.0 1.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 MPI Easy Medium Difficult

15. Turku 2009 – F. Kaftandjieva The Secret of the Hattifatteners 0.0 1.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 MPI OE MC TF

16. Turku 2009 – F. Kaftandjieva All things are so very uncertain, and that’s exactly what makes me feel reassured The Secret of the Hattifatteners: Seek and ye shall find 0.0 1.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 MPI that more research is needed.

17. Turku 2009 – F. Kaftandjieva Moominpappa’s Notes More synergies than tensions! MPI is Group, Skill & Method dependent. More research is needed. Item difficulty and Item format tend to affect MPI. A program for calculating MPI is available on request.

18. Turku 2009 – F. Kaftandjieva Thanks !

19. Turku 2009 – F. Kaftandjieva

20. The Hemulen Who Loved MPI Indices: A Closer Look.97.981.0