Statistical Models for the Analysis of Single-Case Intervention Data Introduction to: Regression Models (models for a single participant) Multilevel Models (models for multiple participants)

Imagine we have multiple cases Cindy George Lucy John

A separate regression could be obtained for each case Or a multilevel analysis could be run

Multilevel Model Multilevel models allow us to answer additional questions: What is the average treatment effect? Does the size of the effect vary across participants? What factors relate to effect size?

What is the average effect for the participants? Baseline Level = γ00

Does the size of the effect vary across participants?

What factors relate to effect size? ADD Non-ADD

Issues to Keep in Mind There still needs to be a match between the trajectory specified in the model and what is seen in the data Effect = b1? This seems incorrect

Correct model specification requires assumptions about multiple error terms Should you assume the errors (eij, r0j, r1j) are independent? Normally distributed?

If one standardizes the effect size, what SD should be used for standardization Within case variance? Between case variance?

Imagine we have multiple studies

Small Sample Size Concerns Multilevel models were developed for large sample size conditions, but single-case applications tend to have a very small number of cases. Given small sample sizes the variances (e.g. variance in the treatment effect across participants) will generally be more poorly estimated than the averages (e.g. the average treatment effect).

Example Analysis Summarize results from 5 studies that examined the effect of intervention on autistic children’s speech DV: Percent intervals with child speech IV: Intervention based on increased parent verbalizations Design: Multiple baseline across participants

Laski, K. E. , Charlop, M. H. , & Schreibman, L. (1988) Laski, K. E., Charlop, M. H., & Schreibman, L. (1988). Training parents to use the natural language paradigm to increase their autistic children’s speech. Journal of Applied Behavior Analysis, 21, 391-400.

One child from Laski et al.

2-Level Model for Laski et al. Parameter Estimated Estimate SE p Fixed Effects Average Baseline Level (θ00) 34.17 8.16 .0030 Average Treatment Effect (θ10) 33.57 5.00 .0001 Variance Components Variance in Baseline Level ( ) 577.8 299.1 .0267 Variance in Treatment Effects ( ) 185.4 110.5 .0468 Covariance u0 & u1 ( ) -294.1 .0838 Variance Within Person ( ) 147.3 18.7 <.0001

Software Code: SAS, R SAS: proc mixed covtest; class Case; model Y= D / solution ddfm=sat; random intercept D / sub=Case type=un; R: twolevel <- lmer(Y ~ D + (1 + D | Case), data2) summary(twolevel)

3-Level Model for All Five Studies

3-Level Model Results Parameter Estimated Estimate SE p Fixed Effects Average Baseline Level (γ000) 18.71 6.31 .0345 Average Treatment Effect (γ100) 31.72 9.37 .0309 Variance Components Between Study Variance in Baseline Level ( ) 128.2 128.3 .1588 Between Study Variance in Treatment Effects ( ) 377.2 320.7 .1198 Covariance v0 & v1 ( ) 165.5 169.8 .3296 Within Study Variance in Baseline Level ( ) 316.9 103.6 .0011 Within Study Variance in Treatment Effects ( ) 222.2 81.8 .0033 Covariance u0 & u1 ( ) -47.8 70.1 .4956 Variance Within Person ( ) 328.7 15.7 <.0001

Software Code: SAS, R SAS: proc mixed covtest; class Study Case; model Y= D / solution ddfm=sat; random intercept D / sub=Study type=un; random intercept D / sub=Case(Study) type=un; R: threelevel <- lmer(Y ~ D + (1 + D | Study:Case) + (1 + D | Study), data3) summary(threelevel)

https://single-case.com/MultiSCED

https://single-case.com/MultiSCED

Conclusion Statistical models (regression and multilevel) provide a flexible approach for estimating treatment effects from single-case data, but care must be taken to ensure the model being used is consistent with the data being analyzed.

Applications and Illustrations Baek, E., & Ferron, J. M. (2013). Multilevel models for multiple-baseline data: Modeling across participant variation in autocorrelation and residual variance. Behavior Research Methods, 45, 65-74. Baek, E. K., Moeyaert, M., Petit-Bois, M., Beretvas, S. N., Van den Noortgate, W., & Ferron, J. M. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24, 590-606. Ferron, J. M., Moeyaert, M., Van den Noortgate, W., & Beretvas, S. N. (2014). Estimating casual effects from multiple-baseline studies: Implications for design and analysis. Psychological Methods, 19, 493-510. Moeyaert, M., Ferron, J., Beretvas, S. N., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52, 191-211. Moeyaert, M., Ugille, M., Ferron, J., Onghena, P., Heyvaert, M., Beretvas, S. N., & Van den Noortgate, W. (2015). Estimating intervention effects across different types of single-subject experimental designs: Empirical illustration. School Psychology Quarterly, 30, 50-63. Rindskopf, D., & Ferron, J. (2014). Using multilevel models to analyze single-case design data. In T. R. Kratochwill & J. R. Levin (Eds.), Single-Case Intervention Research: Statistical and Methodological Advances (pp. 221-246). American Psychological Association. Shadish, W.R., Kyse, E.N., & Rindskopf, D.M. (2013). Analyzing data from single-case designs using multilevel models: new applications and some agenda items for future research. Psychological Methods, 18, 385-405. Van den Noortgate, W., & Onghena, P. (2003). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325-346. Van den Noortgate, W., Onghena, P. (2007). The aggregation of single-case results using hierarchical linear models. The Behavior Analyst Today, 8(2), 196-209. Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental designs. Evidence-Based Communication Assessment and Intervention, 2, 142-151.