1. EPSY 651: Structural Equation Modeling I

2. Where does SEM fit in Quantitative Methodology? Draws on three traditions in mathematics and science: Psychology (Spearman, Kelley, Thurstone, Cronbach, etc. Sociology (Wright) Agriculture and statistics: (Pearson, Fisher, Neymann, Rao, etc.) Largely due to Jöreskog in 1960s & 1970s Map below shows its positioning

4. MANIFEST MODELING Classical statistics within the parametric tradition Canonical analysis subsumes most methods as special cases

5. LATENT MODELING Psychological concept of “FACTOR” is central to latent modeling: unobserved directly but “indicated” through observed variables Emphasis on error as individual differences as well as problem of observation (measurement) rather than “lack of fit” conception in manifest modeling

6. STRUCTURAL EQUATION MODELING PURPOSES MODEL real world phenomena in social sciences with respect to –POPULATIONS –ECOLOGIES –TIME

7. SEM PROCEDURE FOCUS ON DECOMPOSITION OF COVARIANCE MATRIX:  xy =  (  x,  y,  2 x,  2 y,  xy ) +  (e x,e y, e xy ) x =  +  y = By +  x + e

8. TESTING in SEM SEM tests A PRIORI (theoretically specified) MODELS SEM has potential to consider model revisions SEM is not necessarily good for exploratory modeling

9. SEM COMPARISONS SEM can COMPARE Ecologies or Populations for identical models or Simultaneously compare multiple groups or ecologies with each having unique models Statistical testing is available for all parts of all models as well as overall model fit

10. CORRELATION

11. Karl Pearson (1857-1936. (exerpted from E S Pearson, Karl Pearson: An Appreciation of some aspects of his life and works, Cambridge University Press, 1938).

12. Pearson Correlation n  (x i – x x )(y i – y y )/(n-1) r xy = i=1_____________________________ = s xy /s x s y s x s y =  z x i z y i /(n-1) = COVARIANCE / SD(x)SD(y)

13. COVARIANCE DEFINED AS CO-VARIATION COV xy = Sxy “UNSTANDARDIZED CORRELATION” Distribution is statistically workable Basis of Structural Equation Modeling (SEM) is constructing models for covariances of variables

14. SAT Math Calc Grade.364 (40) error. 932(.955) Figure 3.4: Path model representation of correlation between SAT Math scores and Calculus Grades  1 – r 2 s e = standard deviation of errors correlation covariance

15. Path Models path coefficient -standardized coefficient next to arrow, covariance in parentheses error coefficient- the correlation between the errors, or discrepancies between observed and predicted Calc Grade scores, and the observed Calc Grade scores. Predicted(Calc Grade) =.00364 SAT-Math +.5 errors are sometimes called disturbances

16. X Y a XY b X Y e c Figure 3.2: Path model representations of correlation

17. BIVARIATE DATA 2 VARIABLES QUESTION: DO THEY COVARY? IF SO, HOW DO WE INTERPRET? IF NOT, IS THERE A THIRD INTERVENING (MEDIATING) VARIABLE OR EXOGENOUS VARIABLE THAT SUPPRESSES THE RELATIONSHIP? OR MODERATES THE RELATIONSHIP

18. IDEALIZED SCATTERPLOT POSITIVE RELATIONSHIP X Y Prediction line

19. IDEALIZED SCATTERPLOT NEGATIVE RELATIONSHIP X Y Prediction line 95% confidence interval around prediction X. Y.

20. IDEALIZED SCATTERPLOT NO RELATIONSHIP X Y Prediction line

21. SUPPRESSED SCATTERPLOT NO APPARENT RELATIONSHIP X Y Prediction lines MALES FEMALES

22. MODEERATION AND SUPPRESSION IN A SCATTERPLOT NO APPARENT RELATIONSHIP X Y Prediction lines MALES FEMALES

23. IDEALIZED SCATTERPLOT POSITIVE CURVILINEAR RELATIONSHIP X Y Linear prediction line Quadratic prediction line

24. Hypotheses about Correlations

25. One sample tests for Pearson r Two sample tests for Pearson r Multisample test for Pearson r Assumptions: normality of x, y being correlated

26. One Sample Test for Pearson r Null hypothesis:  = 0, Alternate   0 test statistic: t = r/ [(1- r 2 ) / (n-2)] 1/2 with degrees of freedom = n-2

27. One Sample Test for Pearson r ex. Descriptive Statistics for Kindergarteners on a Reading Test (from SPSS) MeanStd. DeviationN Naming letters.5750.328876 Overall reading.6427.241476 Correlations NamingOverall Naming letters1.000.784** Sig. (1-tailed)..000 N7676 Overall reading.784**1.000 Sig. (1-tailed).000. N7676 ** Correlation is significant at the 0.01 level (1-tailed).

28. One Sample Test for Pearson r Null hypothesis:  = c, Alternate   c test statistic: z = (Zr - Zc )/ [1/(n-3)] 1/2 where z=normal statistic, Zr = Fisher Z transform

29. Fisher’s Z transform Zr = tanh -1 r = (1/2) ln[(1+  r  ) /(1 -  r |)] This creates a new variable with mean Z  and SD 1/  1/(n-3) which is normally distributed

30. Non-null r example Null:  (girls) =.784 Alternate:  (girls) .784 Data: r =.845, n= 35 Z  (girls=.784) = 1.055, Zr(girls=.845)=1.238 z = (1.238 - 1.055)/[1/(35-3)] 1/2 =.183/(1/5.65685) = 1.035, nonsig.

31. Two Sample Test for Difference in Pearson r’s Null hypothesis:  1 =  2 Alternate hypothesis  1   2 test statistic: z =( Zr 1 - Zr 2 ) / [1/(n 1 -3) + 1/(n 2 -3)] 1/2 where z= normal statistic

32. Example Null hypothesis:  girls =  boys Alternate hypothesis  girls   2boys test statistic: r girls =.845, r boys =.717 n girls = 35, n boys = 41 z = Z(.845) - Z(.717) / [1/(35-3) + 1/(41- 3)] 1/2 = ( 1.238 -.901) / [1/32 + 1/38] 1/2 =.337 /.240 = 1.405, nonsig.

33. Multisample test for Pearson r Three or more samples: Null hypothesis:  1 =  2 =  3 etc Alternate hypothesis: some  i   j Test statistic:  2 =  w i Z 2 i - w.Z 2 w which is chi-square distributed with #groups- 1 degrees of freedom and w i = n i -3, w.=  w i, and Z w =  w i Z i /w.

34. Example Multisample test for Pearson r Nonsig.

35. Multiple Group Models of Correlation SEM approach models several groups with either the SAME or Different correlations: X X y y boys girls  xy = a

36. Multigroup SEM SEM Analysis produces chi-square test of goodness of fit (lack of fit) for the hypothesis about ALL groups at once Other indices: Comparative Fit Index (CFI), Normed Fit Index (NFI), Root Mean Square Error of Approximation (RMSEA) CFI, NFI >.95 means good fit RMSEA <.06 means good fit

37. Multigroup SEM SEM assumes large sample size, multinormality of all variables Robust as long as skewness and kurtosis are less than  3, sample size is probably > 100 per group (200 is better), or few parameters are being estimated (sample size as low as 70 per group may be OK with good distribution characteristics)

38. Multiple regression analysis

39. The test of the overall hypothesis that y is unrelated to all predictors, equivalent to H 0 :  2 y  123… = 0 H 1 :  2 y  123… = 0 is tested by F = [ R 2 y  123… / p] / [ ( 1 - R 2 y  123… ) / (n – p – 1) ] F = [ SS reg / p ] / [ SS e / (n – p – 1)]

40. Multiple regression analysis SOURCEdfSum of SquaresMean Square F x 1, x 2 …pSS reg SS reg / p SS reg / p SS e /(n-p- 1) e (residual) n-p-1SS e SS e / (n-p-1) total n-1SS y SS y / (n-1)

41. Multiple regression analysis predicting Depression LOCUS OF CONTROL, SELF-ESTEEM, SELF-RELIANCE

42. ss x 1 ss x 2 SSy SSe Fig. 8.4: Venn diagram for multiple regression with two predictors and one outcome measure SS reg

43. Type I ss x 1 Type III ss x 2 SSy SSe Fig. 8.5: Type I contributions SSx 1 SSx 2

44. Type III ss x 1 Type III ss x 2 SSy SSe Fig. 8.6: Type IIII unique contributions SSx 1 SSx 2

45. Multiple Regression ANOVA table SOURCEdfSum of SquaresMean SquareF (Type I) Model2SS reg SS reg / 2SS reg / 2 SS e / (n- 3) x 1 1 SS x1 SS x1 / 1SS x1 / 1 SS e /(n-3) x 21 SS x2  x1 SS x2  x1 SS x2  x1 / 1 SS e /(n-3) e n-3SS e SS e / (n-3) total n-1SS y SS y / (n-3)

46. X1X1 X2X2 Y e  =.5  =.6 r =.4 R 2 =.74 2 +.8 2 - 2(.74)(.8)(.4)  (1-.4 2 ) =.85.387 PATH DIAGRAM FOR REGRESSION

47. Depression DEPRESSION LOC. CON. SELF-EST SELF-REL.471 -.317 -.186 R 2 =.60 e .4

48. Shrinkage R 2 Different definitions: ask which is being used: –What is population value for a sample R 2 ? R 2 s = 1 – (1- R 2 )(n-1)/(n-k-1) –What is the cross-validation from sample to sample? R 2 sc = 1 – (1- R 2 )(n+k)/(n-k)

49. Estimation Methods Types of Estimation: –Ordinary Least Squares (OLS) Minimize sum of squared errors around the prediction line –Generalized Least Squares A regression technique that is used when the error terms from an ordinary least squares regression display non-random patterns such as autocorrelation or heteroskedasticity.ordinary least squares –Maximum Likelihood

50. Maximum Likelihood Estimation Maximum likelihood estimation There is nothing visual about the maximum likelihood method - but it is a powerful method and, at least for large samples, very preciseMaximum likelihood estimation begins with writing a mathematical expression known as the Likelihood Function of the sample data. Loosely speaking, the likelihood of a set of data is the probability of obtaining that particular set of data, given the chosen probability distribution model. This expression contains the unknown model parameters. The values of these parameters that maximize the sample likelihood are known as the Maximum Likelihood Estimatesor MLE's. Maximum likelihood estimation is a totally analytic maximization procedure. MLE's and Likelihood Functions generally have very desirable large sample properties: –they become unbiased minimum variance estimators as the sample size increases –they have approximate normal distributions and approximate sample variances that can be calculated and used to generate confidence bounds –likelihood functions can be used to test hypotheses about models and parameters With small samples, MLE's may not be very precise and may even generate a line that lies above or below the data pointsThere are only two drawbacks to MLE's, but they are important ones: –With small numbers of failures (less than 5, and sometimes less than 10 is small), MLE's can be heavily biased and the large sample optimality properties do not apply Calculating MLE's often requires specialized software for solving complex non- linear equations. This is less of a problem as time goes by, as more statistical packages are upgrading to contain MLE analysis capability every year.

51. Outliers Leverage (for a single predictor): L i = 1/n + (Xi –Mx) 2 /  x 2 (min=1/n, max=1) Values larger than 1/n by large amount should be of concern Cook’s Di =  (Y – Yi) 2 / [(k+1)MSres] –the difference between predicted Y with and without Xi   

52. Outliers In SPSS under SAVE options COOKs and Leverage Values are options you can select Result is new variables in your SPSS data set with the values for each case given You can sort on either one to investigate the largest values for each You can delete the cases with largest values and recompute the regression to see if it changed

53. 635044.03855.01520 425068.02422.04943 415546.02065.02010 565552.01915.02349 566057.01696.01056 413941.01689.02435 773965.01525.01520 526554.01448.01607 393965.01425.02289 304560.01242.01346 536068.01133.03147 525568.01060.00693 553941.01047.00512 428068.00918.02459 598068.00907.01098 486546.00885.00160 t12 t13 t14 COO_1 LEV_1