1. 1 An ordinal IRT model for a circular representation of polytomous data Wijbrandt H. van Schuur University of Groningen 25 th Workshop on Item Response Theory University of Twente October 12-15, 2009

2. 2 Overview A. From dominance model to proximity model and from monotone to circular proximity B. From dichotomous to polytomous data (C. From deterministic to probabilistic model)

3. 3 IRF’s of three dominance items

4. 4 Example dominance model (World Values Study) SubjectsA: HellB: DevilC: HeavenD: God 10000 21000 31100 41110 51111 Do you believe in … Item AHell Item BThe Devil Item CHeaven Item DGod

5. 5 IRF’s of six monotone proximity items

6. 6 Example Monotone proximity model (Electoral compass) itemClintonObamaEdwardsRichard- son McCainHucka -bee RomneyThomsonGiuliani 1110000000 2111110000 3000110000 4000111110 5000011110 6000000111 Item 1The best way to reduce the federal deficit is to increase taxes Item 2 Mortgage lenders should be more tightly controlled Item 3 The US should decrease its spending on defense Item 4Stricter gun control will not reduce crime Item 5 Abortion should be made completely illegal Item 6 The US should never sign international treaties on climate change that limit economic growth

7. 7 7,006,005,004,003,002,00 1,00 0,00 1,00 0,80 0,60 0,40 0,20 0,00 Vertical: probability of positive response Horizontal: items i, j and k scale values (in radians between 0 and 2π) IRF’s of three circular proximity items 0 o 60 o 120 o 180 o 240 o 300 o 360 o = 0 o

8. 8 Larsen, R.J. & Diener (1992), E. Promises and problems with the circumplex model of emotion, p. 31. In: M.S. Clark & J.R. Averill (eds.). Emotion: Review of personality and social psychology (Vol. 13, pp. 25-59), Newbury Park, CA: Sage. Larsen & Diener

9. 9 Brown, M.W. (1992). Circumplex models for correlation matrices. Psychometrika, 57, 470, 479 Brown: Vocational Interests R I A S E C

10. 10 Schwartz: Universals in the content and structure of values Stability Other Self Change Security Conformity Benevolence Universalism Self Direction Stimulation Hedonism Achievement Power Schwartz, S.H. (1992). Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries. In: M.P. Zanna (ed.), Advances in experimental social psychology, Vol. 25 (p. 1-65). San Diego/London: Academic Press..

11. 11 Example circular proximity model (First two dimensions of Big FIVE) Active (N) Lively (NE) Glad (E) Calm (SE) Still (S) Tired (SW) Sad (W) Anxious (NW) 111100000 201110000 300111000 400011100 500001110 600000111 710000011 811000001

12. 12 Violations of deterministic models Dominance model: 1 subject and 2 items: 01-response to item pair (Mokken, 1971) Monotone proximity model: 1 subject and 3 items: 101-response to item triple (Van Schuur, 1984) Circular proximity model: 1 subject and 4 items: 1010- or 0101 response to item quadruple (Leeferink, 1997, Mokken, van Schuur & Leeferink, 2001)

13. 13 Homogeneity (Loevinger) For each elementary scale (pair, triple, quadruple): H = 1 - E(obs)/E(exp) = φ/φ max E(exp): product of relevant probabilities * N (for dominance data) For each item: H i = 1 – Σ E(obs)/Σ E(exp) Summation over all elementary scales that contain item i For whole scale: H = 1 – Σ E(obs)/Σ E(exp) Summation over all elementary scales Person fit: number of elementary scales in response pattern that contain a model violation

14. 14 Exploration Bottom-up hierarchical clustering procedure: 1. “Best” elementary scale 2. “Best” next item

15. 15 Item steps and subject scale values Dominance model: 2 items i and j, 2 categories i: 0 1 1 j: 0 0 1 sum:0 1 2 ──────┬─────┴────┬───┴───┬────── θ 0 δ i01 θ 1 δ j01 θ 2 Proximity models: 3 items i,j,k 2 categories i: 01 1 1 0 0 0 j:00 1 1 1 0 0 k:00 0 1 1 1 0 sum:01 2 3 4 5 6 ───┬─┴─┬─┴──┬─┴─┬─┴─┬─┴─┬┴─┬ δ i01 δ j01 δ k01 δ i10 δ j10 δ k10

16. 16 Item steps and subject scale values Dominance model: 2 items i and j, 2 categories i: 0 1 1 j: 0 0 1 sum:0 1 2 ──────┬─────┴────┬───┴───┬────── θ 0 δ i01 θ 1 δ j01 θ 2 Proximity models: 3 items i,j,k 2 categories i: 01 1 1 2 2 2 j:00 1 1 1 2 2 k:00 0 1 1 1 2 sum:01 2 3 4 5 6 ───┬─┴─┬─┴──┬─┴─┬─┴─┬─┴─┬┴─┬ δ i01 δ j01 δ k01 δ i10 δ j10 δ k10

17. 17 I 01 I 10 J 01 J 10 K 10 K 01 L 01 L 10 1100=2 1110=3 0110=4 0111=5 0011=6 0001=7 1001=0 or 8 1000=1 I J K L Scale values of subjects

18. 18 Scale value of items: dominance model ORDER of the items is generally based on popularity in sample

19. 19 Scale value of items: unfolding model ORDER of the items steps is based on uniqueness of representation (popularity is irrelevant) Which item is middle item: BAC, ABC, or ACB? Requirement for “best” triple: “unique” triple: Positive homogeneity in only one permutation

20. 20 Scale value of items: circumplex model For circular proximity model: a. Each item can be the first in an ordered quadruple: ABCD = BCDA = CDAB = DABC b. Clockwise and counter clockwise: ABCD=DCBA So, arbitrarily beginning with item A: Which item is middle item among remaining three items : CBD, BCD, or BDC? (or quadruples ACBD, ABCD, or ABDC) Requirement for “best” quadruple: “unique” quadruple: Positive homogeneity in only one permutation

21. 21 Polytomous items DominanceMonotone Circular ModelProximity Model Proximity Model Cumulative scaleUnfolding scaleCircumplex scale A B C D E F A B C D E F G A B C D E F G 0 0 0 0 0 0 2 1 1 0 0 0 01 2 1 0 0 0 0 1 0 0 0 0 01 2 1 0 0 0 00 1 2 1 0 0 0 1 1 0 0 0 00 1 2 1 0 0 00 0 1 2 1 0 0 2 2 1 0 0 00 0 1 2 1 0 00 0 0 1 2 1 0 2 2 1 1 0 00 0 0 1 2 1 00 0 0 0 1 2 1 2 2 2 2 1 10 0 0 0 1 2 1 1 0 0 0 0 1 2 2 2 2 2 2 20 0 0 0 1 1 22 1 0 0 0 0 1 For dominance model: Molenaar 1983 For unfolding model: Van Schuur 1993

22. 22 Item steps of polytomous items Dominance model: 2 items i and j, 4 categories i: 0 1 1 2 3 3 3 j: 0 0 1 1 1 2 3 ──┴────┴─────┴────┴───┴──┴─── δ i01 δ j01 δ i12 δ i23 δ j12 δ j23 Proximity models: 1 item i, 4 categories i: 0 1 2 3 2 1 0 ──┴────┴─────┴────┴───┴──┴─── δ i01 δ i12 δ i23 δ i32 δ i21 δ i10

23. 23 Model violations for polytomous dominance data Dominance model: 1 subject and 2 item steps Weight of seriousness of model violation: i: 0 1 1 2 3 3 3 j: 0 0 1 1 1 2 3 ──┴────┴─────┴────┴───┴──┴─── δ i01 δ j01 δ i12 δ i23 δ j12 δ j23 (i=0,j=1) is less serious than (i=0, j=3)

24. 24 Model violations for polytomous unfolding data Monotone Proximity model: Response pattern ABC,323 less bad than ABC,302 Concept of ‘implicit error”: Given ABC,302: AB=30, so C must be 0 and C=1 and C=2 are errors. C=1 is an implicit error, C=2 is the explicit error AC=32, so B must be 2 or 3, and B=1, B=0 in error B=1: implicit; B=0: explicit BC=02, so A must be 0, and A=1, A=2, A=3: error A=1 and A=2: implicit; A=3: explicit Weight of errors in triple: sum of implicit and explicit errors in pairs of triple.

25. 25 Model violations for polytomous circumplex data Circular Proximity model: Response pattern ABCD,3232 less bad than ABCD,3021 Concept of ‘implicit error”: Given ABCD,3021: ABC=302, so D must be 2 or 3; D=1 is the explicit error ABD=301, so C must be 0 or 1, and C=2 explicit error ACD=321, so B must be 2 or 3, and B=1 or B=0 are errors B=0 is explicit error and B=1 is implicit error BCD=021, so A must be 0 or 1, and A=2 or A=3 are errors A=3: explicit and A=2: implicit Weight of errors in quadruple: sum of implicit and explicit errors in triples of quadruple.

26. 26 Homogeneity for polytomous circumplex data For elementary scale: H = 1 – Σ W* E(obs) / Σ W*E(exp) Summation over relevant elementary item step combinations For item or whole scale: H = 1 – ΣΣ W*E(obs) / ΣΣ W*E(exp) + Summation over relevant triples

27. 27 J 1 L 1 K 1 K 2 L 2 K L k(21) k(12) l(12) j(10) l(01) j(21) i(10) I J k(01) I 1 I 2 I 1 J 2 J 1 i(12) i(21) i(01) j(12) L 1 j(01) k(10) l(21) l(10) 1001=0 or 16 0110= 8 2001=1 2101=2 2100= 3 2200= 4 1200= 5 1210= 6 0210= 7 0120= 9 0121=1 0 0021=1 1 0022=1 2 0012=13 1012=1 4 1011=1 5

28. 28 Problems with scale values for subjects 200000unfalsifiable: no model error possible 202020symmetrical: unscalable 101100no highest value (2): ambiguous (change to 202200) Imperfect patterns: calculation clockwise and counterclockwise should give the same result. If not: response pattern is symmetrical (unscalable) or take mean of both values

29. 29 Probabilistic model: Use diagnostic matrices Shape of Correlation matrix (high-low-high values) Similarly: shape of (Conditional) Adjacency matrix shape of Dominance matrix In development: criteria analogous to criteria developed for the Mokken scale by Molenaar and Sijtsma

30. 30 What can we do with circular subject scores? Biologists: compass and clock Mardia (1972): Statistics of directional data Batschelet (1981): Circular Statistics in biology Fisher (1993): Statistical analysis of circular data Compare distributions (uniform, unimodal) Use circular scale values as dependent or independent variable in regression analyses

31. 31 THANK YOU

32. 32 Correlation Matrix: values decrease from the diagonal towards the lowest value (underlined), and then increase again towards the diagonal. A B C D E F G H I A 1.00 0.50 0.07 -0.14 -0.32 -0.29 -0.18 0.07 0.54 B 0.50 1.00 0.57 0.07 -0.18 -0.36 -0.32 -0.14 0.11 C 0.07 0.57 1.00 0.50 0.11 -0.21 -0.32 -0.29 -0.18 D -0.14 0.07 0.50 1.00 0.54 0.14 -0.18 -0.36 -0.32 E -0.32 -0.18 0.11 0.54 1.00 0.54 0.07 -0.18 -0.29 F -0.29 -0.36 -0.21 0.14 0.54 1.00 0.54 0.07 -0.18 G -0.18 -0.32 -0.32 -0.18 0.07 0.54 1.00 0.54 0.07 H 0.07 -0.14 -0.29 -0.36 -0.18 0.07 0.54 1.00 0.54 I 0.54 0.11 -0.18 -0.32 -0.29 -0.18 0.07 0.54 1.00

33. 33 Conditional Adjacency Matrix: Response value (>=): 1 A B C D E F G H I A 1.00 0.72 0.52 0.45 0.34 0.38 0.41 0.52 0.76 B 0.78 1.00 0.78 0.56 0.41 0.33 0.33 0.41 0.56 C 0.56 0.78 1.00 0.78 0.56 0.41 0.33 0.33 0.41 D 0.45 0.52 0.72 1.00 0.76 0.59 0.41 0.31 0.34 E 0.36 0.39 0.54 0.79 1.00 0.79 0.54 0.39 0.36 F 0.38 0.31 0.38 0.59 0.76 1.00 0.76 0.52 0.41 G 0.43 0.32 0.32 0.43 0.54 0.79 1.00 0.75 0.54 H 0.56 0.41 0.33 0.33 0.41 0.56 0.78 1.00 0.78 I 0.79 0.54 0.39 0.36 0.36 0.43 0.54 0.75 1.00

34. 34 Dominance Matrix: Response value (>=): 1 A B C D E F G H I A 0.00 0.14 0.25 0.29 0.34 0.32 0.30 0.25 0.13 B 0.11 0.00 0.11 0.21 0.29 0.32 0.32 0.29 0.21 C 0.21 0.11 0.00 0.11 0.21 0.29 0.32 0.32 0.29 D 0.29 0.25 0.14 0.00 0.13 0.21 0.30 0.36 0.34 E 0.32 0.30 0.23 0.11 0.00 0.11 0.23 0.30 0.32 F 0.32 0.36 0.32 0.21 0.13 0.00 0.13 0.25 0.30 G 0.29 0.34 0.34 0.29 0.23 0.11 0.00 0.13 0.23 H 0.21 0.29 0.32 0.32 0.29 0.21 0.11 0.00 0.11 I 0.11 0.23 0.30 0.32 0.32 0.29 0.23 0.13 0.00

35. 35 Score group matrix. Response value (>=): 1 score N scale A B C D E F G H I group range 1 6 (17- 0) 1.00 0.67 0.33 0.00 0.00 0.17 0.50 0.83 1.00 2 6 ( 1- 2) 1.00 1.00 0.67 0.33 0.00 0.00 0.17 0.50 0.83 3 6 ( 3- 4) 0.83 1.00 1.00 0.67 0.33 0.00 0.00 0.17 0.50 4 6 ( 5- 6) 0.50 0.83 1.00 1.00 0.67 0.33 0.00 0.00 0.17 5 10 ( 7- 9) 0.20 0.40 0.70 1.00 1.00 0.80 0.40 0.10 0.10 6 10 (10-12) 0.10 0.00 0.20 0.60 0.80 1.00 0.90 0.50 0.20 7 6 (13-14) 0.33 0.00 0.00 0.17 0.50 0.83 1.00 1.00 0.67 8 6 (15-16) 0.67 0.33 0.00 0.00 0.17 0.50 0.83 1.00 1.00

36. 36 Exploration Bottom-up hierarchical clustering procedure: 1. “Best” elementary scale 2. “Best” next item Ad 1: - High(est) homogeneity - highest number of subjects who use items of elementary scale in acceptable pattern (for proximity models) - unique representation (for proximity models) Ad 2: - High(est) homogeneity - unique representation (for proximity models)