# Sample Size and Power for CFA: A Monte Carlo Approach

## Presentation on theme: "Sample Size and Power for CFA: A Monte Carlo Approach"— Presentation transcript:

Sample Size and Power for CFA: A Monte Carlo Approach
Myers, N.D., Ahn, S., & Jin, Y.

I. Purpose Demonstrate how Monte Carlo methods can be used to make empirically-based decisions: N for a fixed level of π, and, (i.e., N?) π for a fixed N (π?) under a CFA model with model-data conditions commonly encountered in sport & exercise Generality of MC methods has allowed for two types of applications in statistics1 MC studies of statistical methods MC methods in data analysis

Why is the additional complexity worthwhile?
I. Purpose Why is the additional complexity worthwhile? Rules of thumb are known to be of limited utility2 N ≥ 200 N/p ≥ 10 N/q ≥ 5 construct reliability and adequate N Adequate N for CFA depends on many factors that typically vary across any two studies that use real data and inexact theoretical models These factors can be directly modeled using MC methods

Informal review of RQES (and others)
I. Purpose Informal review of RQES (and others) Some observed trends in practice: single population models many p and multiple h ordinal data a sig c2 value and e ≤ .05 variable L and Y (parameters of interest, qi) few vc > 1 and/or off-diagonal Q ≠ 0 A priori plan for N, for a desired level of p? An estimate of p, for a fixed N? ....must have “knowns” Model the messiness of practice… N? p?3

II. A Conceptual Demonstration
Coaching Efficacy: coach’s belief in his/her ability to influence the learning and performance of his/her athletes4 Psych: Bandura (1997)5; Educ: Denham & Michael (1981)6 Coaching Effectiveness: Horn (2002)7

II. A Conceptual Demonstration
Coaching Efficacy Scale II-HST8

III. Methods/Results Design Stage: Conceptualize experiment - “Intro” 9 Research Question(s) Smallest N necessary to achieve at least .80 p for each qi Given a particular N , what is the p estimate for each qi 2. Derive theoretical model & population model 3. Design Experiment Model mis-specification (MODEL POPULATION versus MODEL) N = 800, 200, 300, 400, 500; NR = 10,000 Ordinal data 4. Choose values of population parameters range of parameter estimates 5. Choosing Software Mplus 5.2 (code is available by request)10,11

III. Methods/Results Generating Data Stage: Performing experiment
“Data collection” 6. Executing the simulations Same code varied N 7. File storage Saved where input file is located (stuff happens…) 8. Trouble shooting Data generation problem (e.g., no obs in a particular category) Data analytic problem (e.g., non-convergent, improper solution)

III. Methods/Results Interpreting Results Stage: Findings from experiment “Results/Discussion” 9. Summarizing Results Theoretical Model fit to data generated from Population Model Q1 (N?): A relatively small N (200) provides ample π (> 98%) Q2 (p?): common N (300, 400, 500) is similar to Q1 (π > 99%) Problematic bias values (> |10%|) and coverage values (< 91%), for a few parameters (~20%) are observed for all sample sizes Population Model fit to data generated from Population Model To what degree might the “relatively minor” model mis-specifications be responsible for these problems? Q1 (N?): and Q2 (p?) similar to findings from Theoretical Model except no problems with bias values and coverage values

IV. Importance Thank you. Ms is in press at RQES.
MC methods have long been used to advance statistical theory. There have been several recent calls to use Monte Carlo methods as a tool to improve applications of quantitative methods in substantive research. Mplus code is available and can easily be altered CES II – HST: N ≥ 200 for the theoretical model N ≥ 300 for the population model a level of misfit that may be regarded as trivial in practice may have troubling effects on parameters of conceptual interest encourage sustained efforts toward generating closer approximations of population models Thank you. Ms is in press at RQES.

Gentle, J. E. (2003). Random number generation and Monte Carlo methods (2nd ed.). New
York: Springer. Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral Research, 33, 3. MacCallum, R. (2003). Working with imperfect models. Multivariate Behavioral Research, 38, Feltz, D.L., Chase, M.A., Moritz, S.E., & Sullivan, P.J. (1999). A conceptual model of coaching efficacy: Preliminary investigation and instrument development. Journal of Educational Psychology, 91, 5. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman. 6. Denham, C.H., & Michael, J.J. (1981). Teacher sense of efficacy: A definition and a model for further research. Educational Research Quarterly, 5, 7. Horn, T.S. (2002). Coaching effectiveness in the sport domain. In T.S. Horn (Ed.), Advances in sport psychology (2nd ed., pp ). Champaign, IL: Human Kinetics. 8. Myers, N. D., Feltz, D. L., Chase, M. A., Reckase, M. D., & Hancock, G. R. (2008). The Coaching Efficacy Scale II - High School Teams. Educational and Psychological Measurement, 68, 9. Paxton, P., Curran, P. J., Bollen, K. A., Kirby, J., & Chen, F. (2001). Monte Carlo experiments: Design and implementation. Structural Equation Modeling, 8, 10. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 11. Muthén, L.K. and Muthén, B.O. ( ). Mplus User’s Guide (6th ed.). Los Angeles, CA: Muthén & Muthén.

II. A Conceptual Demonstration
Coaching Efficacy Scale II-HST 8

Similar presentations