1. Sample Size and Power for CFA: A Monte Carlo Approach
Myers, N.D., Ahn, S., & Jin, Y.

2. I. Purpose
Demonstrate how Monte Carlo methods can be used to make empirically-based decisions:
N for a fixed level of π, and, (i.e., N?)
π for a fixed N (π?)
under a CFA model with model-data conditions commonly encountered in sport & exercise
Generality of MC methods has allowed for two types of applications in statistics1
MC studies of statistical methods
MC methods in data analysis

3. Why is the additional complexity worthwhile?
I. Purpose
Why is the additional complexity worthwhile?
Rules of thumb are known to be of limited utility2
N ≥ 200
N/p ≥ 10
N/q ≥ 5
construct reliability and adequate N
Adequate N for CFA depends on many factors that typically vary across any two studies that use real data and inexact theoretical models
These factors can be directly modeled using MC methods

4. Informal review of RQES (and others)
I. Purpose
Informal review of RQES (and others)
Some observed trends in practice:
single population models
many p and multiple h
ordinal data
a sig c2 value and e ≤ .05
variable L and Y (parameters of interest, qi)
few vc > 1 and/or off-diagonal Q ≠ 0
A priori plan for N, for a desired level of p?
An estimate of p, for a fixed N?
....must have “knowns”
Model the messiness of practice… N? p?3

5. II. A Conceptual Demonstration
Coaching Efficacy: coach’s belief in his/her ability to influence the learning and performance of his/her athletes4
Psych: Bandura (1997)5; Educ: Denham & Michael (1981)6
Coaching Effectiveness: Horn (2002)7

6. II. A Conceptual Demonstration
Coaching Efficacy Scale II-HST8

7. III. Methods/Results
Design Stage: Conceptualize experiment - “Intro” 9
Research Question(s)
Smallest N necessary to achieve at least .80 p for each qi
Given a particular N , what is the p estimate for each qi
2. Derive theoretical model & population model
3. Design Experiment
Model mis-specification (MODEL POPULATION versus MODEL)
N = 800, 200, 300, 400, 500; NR = 10,000
Ordinal data
4. Choose values of population parameters
range of parameter estimates
5. Choosing Software
Mplus 5.2 (code is available by request)10,11

8. III. Methods/Results Generating Data Stage: Performing experiment
“Data collection”
6. Executing the simulations
Same code varied N
7. File storage
Saved where input file is located (stuff happens…)
8. Trouble shooting
Data generation problem (e.g., no obs in a particular category)
Data analytic problem (e.g., non-convergent, improper solution)

9. III. Methods/Results
Interpreting Results Stage: Findings from experiment
“Results/Discussion”
9. Summarizing Results
Theoretical Model fit to data generated from Population Model
Q1 (N?): A relatively small N (200) provides ample π (> 98%)
Q2 (p?): common N (300, 400, 500) is similar to Q1 (π > 99%)
Problematic bias values (> |10%|) and coverage values (< 91%), for a few parameters (~20%) are observed for all sample sizes
Population Model fit to data generated from Population Model
To what degree might the “relatively minor” model mis-specifications be responsible for these problems?
Q1 (N?): and Q2 (p?) similar to findings from Theoretical Model except no problems with bias values and coverage values

10. IV. Importance Thank you. Ms is in press at RQES.
MC methods have long been used to advance statistical theory. There have been several recent calls to use Monte Carlo methods as a tool to improve applications of quantitative methods in substantive research.
Mplus code is available and can easily be altered
CES II – HST: N ≥ 200 for the theoretical model
N ≥ 300 for the population model
a level of misfit that may be regarded as trivial in practice may have troubling effects on parameters of conceptual interest
encourage sustained efforts toward generating closer approximations of population models
Thank you.
Ms is in press at RQES.

11. Gentle, J. E. (2003). Random number generation and Monte Carlo methods (2nd ed.). New
York: Springer.
Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998). Is more ever too much? The
number of indicators per factor in confirmatory factor analysis. Multivariate Behavioral
Research, 33,
3. MacCallum, R. (2003). Working with imperfect models. Multivariate Behavioral Research, 38,
Feltz, D.L., Chase, M.A., Moritz, S.E., & Sullivan, P.J. (1999). A conceptual model of coaching efficacy:
Preliminary investigation and instrument development. Journal of Educational Psychology, 91,
5. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman.
6. Denham, C.H., & Michael, J.J. (1981). Teacher sense of efficacy: A definition and a model for further research. Educational Research Quarterly, 5,
7. Horn, T.S. (2002). Coaching effectiveness in the sport domain. In T.S. Horn (Ed.), Advances in sport psychology (2nd ed., pp ). Champaign, IL: Human Kinetics.
8. Myers, N. D., Feltz, D. L., Chase, M. A., Reckase, M. D., & Hancock, G. R. (2008). The
Coaching Efficacy Scale II - High School Teams. Educational and Psychological
Measurement, 68,
9. Paxton, P., Curran, P. J., Bollen, K. A., Kirby, J., & Chen, F. (2001). Monte Carlo experiments:
Design and implementation. Structural Equation Modeling, 8,
10. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample
size and determine power. Structural Equation Modeling, 9,
11. Muthén, L.K. and Muthén, B.O. ( ). Mplus User’s Guide (6th ed.). Los Angeles, CA:
Muthén & Muthén.

16. II. A Conceptual Demonstration
Coaching Efficacy Scale II-HST 8