1. Confirmatory Factor Analysis Psych 818 DeShon

2. Construct Validity: MTMM ● Assessed via convergent and divergent evidence ● Convergent – Measures of the same construct should correlate highly ● Divergent – Measures of different constructs should not correlate highly

3. MTMM Example ● Byrne & Goffin (1993) ● 158 11 th graders ● 4 traits measured using 4 methods – 16 indicators

4. MTMM Example

7. ● Based on the comparison of nested models – Freely correlated traits and methods – No traits – freely correlated methods – Perfectly correlated traits – freely correlated methods – Freely correlated traits – perfectly correlated methods

8. Correlated traits and methods

9. No traits / Correlated Methods

10. Perfectly Correlated Traits / Freely Correlated Methods

11. Freely Correlated Traits / Perfectly Correlated methods

12. MTMM Example: Model Summary

13. Convergent Validity - Traits

14. Discriminant Validity - Traits

15. Discriminant Validity - Methods

16. MTMM Summary ● Comparisons of specific CFA models that are consistent with the MTMM logic provide practical and statistical evidence of convergent and divergent validity. ● Model comparisons used to examine the function of both traits and methods ● This model sometimes fails to converge or has inadmissable results – Needs at least 3 traits and methods to be identified ● If so, use a correlated uniqueness approach – Kenny (1976), Marsh (1989)

17. Hierarchical CFA ● Just as latent variables might explain correlation among items, second order latent variables might explain correlation among factors

18. Hierarchical CFA Quality of life for adolescents: Assessing measurement properties using structural equation modelling Meuleners, Lee, Binns & Lower (2003). Quality of Life Research, 12, 283– 290.

19. 2 nd Order CFA

20. Hierarchical CFA Depression (CES-D ) Positive Affect Negative Affect Somatic Symptoms HappyEnjoyBotheredBluesDepressedSadMindEffortSleep.795 a.882 a.810 a Model Fit Statistics: N= 868, χ 2 (26)= 68.690, p<.001, SRMR=.055, IFI=.976 a Second-order loadings were set equal for empirical identification. All loadings significant at p <.001.

21. Residualized 2 nd Order CFA

22. Hierachical CFA ● Scaling the latent variables – 1 st order latent variables are scaled by constraining one path to a manifest variable to 1.0 – 2 nd order latent variables are scaled by setting their variances to 10.0 (standardized)

23. Measurement Invariance ● A precondition of comparing groups on a construct (e.g., efficacy, intelligence, personality) is that the measure of the construct functions the same across groups. ● If the measure doesn't function equivalently across groups, you're done! – No group comparisons are meaningful ● Also relevant to longitudinal studies – The measure must “mean” the same thing over time

24. Measurement Invariance ● The dominant approach to this issue is Meredith's (1993) model of factorial invariance ● Basically compare the equality of CFAs across groups – Aka – simultaneous factor analysis in several populations (SIFASP)

25. Types of Measurement Invariance ● Meredith (1964; 1993) ● Factorial Invariance

26. Invariance logic ● Evaluate a single factor model across populations ● Questions: – Is the basic pattern of parameter estimates the same across groups? – Are the factor loadings equal across groups? – Are the indicator intercepts the same across groups? – Are the error variances equal across groups? – Are the latent means and variances equal across groups? ● This last issue may not mean anything about invariance

27. Multi-Sample CFA with Structured Means ● Must estimate the intercepts for the regressions of the manifest variables to the latent variable(s) ● Byrne (2004). Testing measurement invariance in AMOS. SEM, 11, 272-300.

28. 1 st Step ● Invariance of population covariance matrices – If you find this, then it is reasonable to assume that the underlying model is the same

29. Configural Invariance ● Relaxed constraints requiring only the same number and pattern of factor loadings – factor-variable regressions need not be the same across groups (Horn, McArdle, & Mason, 1983). ● Evidence for qualitative similarity across groups but does not permit quantitative comparisons.

30. Weak Factorial or "Metric" Invariance ● Equality constraints on the factor-variable regressions across groups while ensuring the factor variances and covariances are free to vary ● minimally necessary condition for establishing invariant measurement across groups (Horn, 1991; Horn, McArdle, & Mason, 1983; Meredith, 1964; 1993) ● The factor loadings are proportionally equivalent to corresponding loadings in other groups since the factor variances and covariances in each group must be free to vary across groups

31. Strong Factorial Invariance ● Strong factorial invariance requires the additional constraint of invariant mean intercepts across groups – All mean differences in the variables are expressed at the factor level – Factor mean differences across groups are expressed as differences relative to an arbitrary group (where factor means are set to zero or some arbitrary value; Sorbom, 1974; see also Horn & McArdle, 1992). ● Group differences in unique means indicates the presence of additive bias

32. Strict Factorial Invariance ● Additional constraint of invariant error variances as well as unique means and factor loadings. ● Test of whether there are differences in amount of specific/error variance. ● This model forces the combined specific and random error components of each variable to be equivalent across groups such that differences in variance across groups are permitted only at the latent variable level.

33. Strict Factorial Invariance ● Under strict factorial invariance the measurement models do not differ across the groups and can, in principle, be collapsed into one group.

34. Evaluating Factorial Invariance ● Equal covariance matrices? ● Configural – Same pattern of factor loadings ● Weak (“Metric”)Factorial Invariance – Factor loadings equal ● Strong Factorial Invariance – Factor loadings equal – Mean intercepts equal ● Strict Factorial Invariance – Factor loadings equal – Mean intercepts equal – Unique variances equal

35. Example

36. What to do? ● If strict factorial invariance, or at least strong invariance, does not hold, comparisons across groups cannot be made. ● Drop items? ● Covariates?