Comments on David Kaplan’s Representing a Research Problem As a Structural Equation Model Jelte M. Wicherts Psychological Methods Department University of Amsterdam

Possibilities of SEM in experimental studies The use of SEM enables better tests of theoretical predictions and assumptions Higher degree of falsifiability However, in psychology SEM is rarely used in experimental studies

Experimental effect of exposure therapy on arachnophobia Anxiety questionnaire Frequency of avoidance Respiration during exposure Exposure Therapy MANOVA: significant multivariate effect for condition

Experimental effect of therapy on arachnophobia Anxiety questionnaire Frequency of avoidance Respiration during exposure Exposure Therapy A test for measurement invariance across conditions using a 1-factor model provides a test of full mediation of effect by the arachnophobia-factor Arachnophobia

What is the value added in using SEM? –Testing mediation (the main value) –Greater degrees of falisfiability (d.f.) –Handling of measurement error Although not restricted to SEM –Incredible model flexibility Yes: SEM has added value With SEM much more can be learned from your data

But why is there a reluctance to use advanced SEM? 1)Sample size, computer programs 2)Why would one use more strict tests of theoretical predictions? 3)It should not be too difficult! 4)Knowledge of models among substantive researchers 5)Theories/predictions are often vague

2) Why would one use more strict tests of theoretical predictions? Higher DF means higher falsifiability This means more severe empirical test of theoretical predictions However, the use of restricted SEMs also means raising the empirical bar ?

3) It should not be too difficult! It is often difficult to get papers with a modeling perspective published in substantive journals Typical reviewer’s reaction: “this paper has too much greek and is more suited for a specialized statistical journal like psychometrika” Also: papers with a formal stance often have little impact

3) It should not be too difficult! Impact of empirical papers published in Journal of Personality and Social Psychology in 2000 and Average Impact Use of Equations? No19.11 (SD=16.1) Yes10.83 (SD=9.10) Mann-Whitney test: Z=1.80, p<.05 (one-sided)

4) Knowledge of models Substantive researchers are often unfamiliar with possibilities (and the workings!) of SEM

5) Theories are often vague In psychology many theories are verbal and not very specific (more/less laws). It is often very difficult to translate theoretical predictions to SEMs. E.g., people with high self-esteem benefit more from exposure therapy

Some solutions… Sample size, computer programs Why would one use more strict tests of theoretical predictions? It should not be too difficult! Knowledge of models among substantive researchers Theories/predictions are often vague -> Robustness studies -> Stress advantages: the use of SEM can solve issues! -> Write tutorials & better education -> Write clear papers! -> Join editorial boards -> More (& earlier) involvement in substantive research

Good examples of research with SEM will hopefully lead to an increase in the valid use of SEM