1. Assessing Hypothesis Testing Fit Indices
Kline Chapter 8 (Stop at 210)
Beaujean Appendix A

2. Fit Indices It’s complicated yo. There are a lot of them.
They do not imply that you are perfectly right.
They have guidelines but they are not perfect either.
People misuse them.
Etc.

3. Fit Indices Limitations:
Fit statistics indicate an overall/average model fit.
That means there can be bad sections, but the overall fit is good.
No one magical number/summary.
They do not tell you where a misspecification occurs.

4. Fit Indices Limitations
Do not tell you the predictive value of a model.
Do not tell you if it’s theoretically meaningful.

5. Fit Indices What size? I need a rule?!
Everyone cites Hu and Bentler (1999) for the golden standards.
Same problem that Cohen had (we love rules).
So when the fit is messy, cite Kline (page 197) as reasons that’s not a bad thing
This section is an interesting read, especially if you have trouble publishing, but not crucial to your understanding of fit indices

6. Fit Indices
Model test statistic – examines if the reproduced correlation matrix matches the sample correlation matrix
Sometimes called “badness of fit”
Want these to be small

7. Fit Indices Traditional NHST = reject-support context
You reject the null to show your research hypothesis is correct.
SEM Hyp Testing = accept-support context
You do not reject the null showing that your model is consistent with the population

8. Model Test Statistic Chi-square Formula = (N-1)FML
FML = is the minimum fit function in ML estimation
P values are based on df for your model and a chi-square distribution
You want this to be nonsignificant.
But this is a catch 22!

9. Model Test Statistic Chi-square is biased by
Multivariate non-normality
Correlation size – bigger correlations can be bad for you (harder to estimate all that variance)
Unique variance
Sample size

10. Model Test Statistic
Everyone reports chi-square, but people tend to ignore significant values
(I’m sort of eh on his YOU MUST PAY ATTN OR DIE talk in this section)

11. Model Test Statistic
Normed chi-square or (X2/df) – this used to be widely reported and used
The criterion was < 3.00 were good models
Now most people have moved away from this procedure (i.e. don’t use this one).
Also considered an absolute fit index.

12. Model Test Statistic
Since X2 is biased by a bunch of things, you can use a robust estimator to help fix that bias.
Satorra-Bentler
Yuan-Bentler
See page 157 for all the different robust options

13. Fit Indices
Many of the statistics described next are compared to the following model types:
Independence – a model assuming no relationships between the variables (i.e. parameters are not significant)
Saturated – a model assuming all parameters exist (i.e. df = 0).

14. Fit Indices
Both types of statistical inferences have their problems … and especially in SEM because it is easy to find statistics that you would normally reject, even with good model fit.
Tends to be too black and white (reject or not to reject!)

15. Fit Indices Alternative Fit Indices
Not traditionally a dichotomous yes-no decision
Do not distinguish between sampling error and evidence against the model

16. Fit Indices Alternative Fit Indices Absolute Fit Indices
Incremental Fit Indices
Parsimony-adjusted Index
Predictive Fit Indices

17. Fit Indices Absolute fit indices
Proportion of the covariance matrix explained by the model
You can think about these as sort of R2
Want these values be high

18. GFI Do not use this sucker unless you want to get a nasty review.
GFI, AGFI, PGFI
Lots of research showing it’s positively biased

19. SRMR Standardized root mean (square) residual Want small values
Excellent < .06 (not a typo different than book)
Good < .08
Acceptable < .10
Eeek > .10

20. Fit Indices Incremental Fit Indices
Also known as comparative fit indices
Compared to the improvement over the independence model (remember that’s the one with no relationships between the variables)
Not necessarily the best indices

21. CFI Comparative Fit Index
Values are 0 to 1 (sometimes you’ll get slightly over 1, usually indicates something is wrong)
Want high values
Excellent >.95
Good > .90
Blah < .90
All the following FIs have the same cut off rules

22. NFI
Normed Fit Index
A variation of the CFI, as it was said to underestimate for small samples
1 – (X2baseline / X2model)

23. IFI Incremental Fit Index Also known as Bollen’s Non-normed fit index
Modified NFI that doesn’t depend on sample size so much.
(X2baseline - X2model) / (X2baseline - dfmodel)

24. RFI Relative Fit Index Also known as Bollen’s Normed Fit Index
(X2model/dfmodel) / (X2baseline/dfbaseline)

25. TLI
Tucker Lewis Index
Also known as the Bentler-Bonet Non-Normed Fit Index
( (X2model/dfmodel) - (X2baseline/dfbaseline) ) / ( (X2baseline/dfbaseline) – 1)

26. Fit Indices Parsimony-adjusted index
These include penalties for model complexity
Normally more paths = better fit.
These will have smaller values for simpler models

27. RMSEA Root mean squared error of approximation
Parsimony-adjusted index
Want small values
Excellent < .06 (not a typo different than book)
Good < .08
Acceptable < .10
Eeek > .10
Report Confidence Interval!

28. Pclose Tests if the RMSEA is in the excellent range
You want p > .50 to show that there is a high probability that RMSEA is effectively zero

29. Fit Indices Predictive fix indices
Estimate model fit in a hypothetical replication of the study with the same sample size randomly drawn from the population
Not always used
Often used for model comparisons, see below.
Often also consider parsimony adjusted indices.

30. Model Comparisons Let’s say you want to adjust your model
You can compare the adjusted model to the original model to determine if the adjustment is better
Let’s say you want to compare two different models
You can compare their fits to see which is better

31. Model Comparisons Nested models
If you can create one model from another by the addition or subtraction of parameters, then it is nested
Model A is said to be nested within Model B, if Model B is a more complicated version of Model A.
For example, a one-factor model is nested within a two-factor as a one-factor model can be viewed as a two-factor model in which the correlation between factors is perfect).

32. Nested Models Chi-square difference test
| Subtract Model 1 X2 – Model 2 X2|
Subtract Model 1 df – Model 2 df
Use a chi-square table to look up p < .05 for difference in df
See if the first step is greater than that value
If yes, you say the model with the lower chi-square is better
If no, you say they are the same and go with the simpler model

33. Nested Models CFI difference test Subtract CFI model 1 – CFI model 2
If the change is more than .01, then the models are considered different
This version is not biased by sample size issues with chi-square.

34. Nested Models So how can I tell what to change?
NOTE: JUST CHANGE ONE THING AT A TIME!
Use modification indices!
They tell you what the chi-square change would be if you add the path suggested.
Based on X2(1) – called a Lagrange Multiplier
Remember that p < .05 = 3.84

35. Nested Models Can be tested with lavaan using the anova() function.
More on this in the multigroup section

36. Non-Nested Models AIC – Akaike Information Criterion
BIC – Bayesian Information Criterion
SABIC – Sample size Adjusted Information Criterion
All of these penalize you for having more complex models. If all other things are equal, it is biased to the simpler model.

37. Non-Nested Models
All of the ICs are how much the sample will cross validate in the future
You want them to be small, so you pick the smallest one of the two models (how different?)

38. Non-Nested Models ECVI – expected cross validation index
Fmin + (2t / ( n – p – 2) )
T number of parameters estimated
P number of squares
Again, you want small values, so you pick the model with the smallest ECVI

39. OMG! So what to do? Mainly people report: X2(df), RMSEA, SRMR, CFI
Determine the type of model change to use the right model comparison statistic

40. Example Two models Let’s make the models
And compare them with their fit indices
Are they nested or not?
See handout on blackboard.