1. Analyzing Intervention Studies
Shaun M. Eack, Ph.D.
University of Pittsburgh
July 14, 2011

2. Overview Analytic Principles Analytic Goals and Methods
Fixed-effects models
Mixed-effects models
Reporting (CONSORT)

3. Analytic Principles of Intervention Studies
Trial Registration ( – all clinical trials funded by the NIH and other federal agencies must be registered
Required for publication in high-impact journals (Archives, NEJM, JAMA)
Investigators are held to the details outlined in the registration, particularly for analysis and outcomes

4. Analytic Principles of Intervention Studies
Primary outcomes
1 or 2 key outcome measures
Simple and easy to understand
Secondary outcomes
Additional exploratory measures
Not necessarily powered to detect in your study

5. Analytic Principles of Intervention Studies
Completer Analysis – Examining only those who complete the study
Partial Completer Analysis – Examining those who receive a specified minimum amount of treatment exposure
Intent-to-treat analysis – Examining all individuals who you intended to treat

6. Methods of Handling Attrition
Ignore it (completer analysis)
Impute
Mean
Regression
Multiply Impute
LOCF
Estimate
Expectation-Maximization/Maximum Likelihood

7. Goals of Analysis Purposes Examples
Examine change in averages over time
Examine differences in averages between groups
Examine differences in changes in averages over time between groups
Examples
Pretest-Posttest, therapy and depression
Posttest Only, aspirin and heart attack
RCT, disulfiram and alcohol use

8. Pretest-Posttest, Therapy and Depression

9. Posttest Only, Aspirin and Heart Attack

10. RCT, Disulfiram and Alcohol Abuse (no placebo)

11. RCT, Disulfiram and Alcohol Abuse (no placebo)

12. Fixed-Effects Models
Independent T-Test - compare averages of 2 groups at 1 time point
Analysis of Variance (ANOVA) - compare averages between 2 or more groups at 1 time point
Dependent T-Test - compare averages of 1 group at 2 time points
Repeated Measures ANOVA - compare averages between 2 or more groups at 2 or more time points

13. Fixed-Effects Models and Designs
Independent T-Test
Posttest Only, Static-Group Comparison
Dependent T-Test
Time Series, Repeated Treatment, One-Group Pretest-Posttest
ANOVA
Posttest Only and Static-Group Comparison with more than 2 groups
Repeated Measures ANOVA
Classic Experiment, Active Control Group, Placebo Control Group, Nonequivalent Comparison Groups, Multiple Time Series

14. Mixed-Effects Models Some effects are fixed
Average intercept for the sample
Average slope for the sample
Some effects are random and allowed to vary across individuals
Random (individual) deviations from the mean intercept
Random (individual) deviations from the mean slope

16. Mixed-Effects Models
Mixed-effects models of longitudinal data are a special type of hierarchical linear modeling – Growth Curve Modeling
Considered HLM because of the multilevel nature of data examined
Level 1 – Individual measurements over time
Level 2 – Constant individual characteristics
Level 1 is nested within Level 2 (measurement occasions nested within individuals)

17. Unconditional Growth Model
Level 1:
Yti = β0i + β1i(Time)t + rti (1)
Level 2:
β0i = γ00 + μ0i (2)
β1i = γ10 + μ1i (3)
β0i - Intercept for person i
β1i - Slope for person i
γ00 - Average intercept
γ10 - Average slope

18. Conditional Growth Model
Level 1:
Yti = β0i + β1i(Time)t + rti (4)
Level 2:
β0i = γ00 + γ01(Treatment)i + μ0i (5)
β1i = γ10 + γ11(Treatment)i + μ1i (6)
Treatment: 0 = Control; 1 = Therapy
γ01 - Effect of treatment on intercept
γ11 - Effect of treatment on slope

19. Conditional Growth Model
Level 1:
Yti = β0i + β1i(Time)t + rti (7)
Level 2:
β0i = γ00 + γ01(Treatment)i + μ0i (8)
β1i = γ10 + γ11(Treatment)i + μ1i (9)
Mixed Model:
Yti = γ00 + γ01(Treatment)i + γ10(Time)t + γ11(Treatment)i(Time)t + μ0i + μ1i(Time)t + rti (10)

20. Coding Time
The coding of the time variable is very important for interpretation
Baseline = 0, Month 9 = 1, Month 18 = 2
Intercept = average score on outcome Y at baseline
Baseline = 2, Month 9 = 1, Month 18 = 0
Intercept = average score on outcome Y at 18 months
Slope = change is linear
Baseline = 0, Month 9 = 1, Month 18 = 4
Slope = change is quadratic

21. Advantages of Growth Modeling
Automatically estimates missing data
Includes individuals who are missing time points
Does not require equal timepoints for all participants
Allows correlated lagged residuals
Capable of modeling heterscedasticity

22. Disadvantages of Growth Modeling
Requires larger number of observations
Obs > 50
Obs = N * T
Feasible with large N or T
Estimation is more difficult
Easy to ask questions you do not intend
Generally more conceptually complex

23. Multiple Inference Testing
Probability of 1/10 tests reaching < .05 is 50%. Odds increase as you conduct more tests.
Adjustments have to be made:
Test only a small number of primary outcomes first
Apply a correction for inflated Type I error

24. Corrections for Multiple Inference Testing
Bonferroni – p.crit = p.obs / k tests
Low power
Sacrifices Type II error
20 tests, p.crit = .05/20 = .0025
Ignores correlations among tests
Ignores decreased probability of Type I error with multiple lower p-values

26. Corrections for Multiple Inference Testing
Hochberg (1988)
Benjamini and Hochberg (1995)
False Discovery Rate
Nothing trumps a good hypothesis and plan

27. Reporting CONSORT – Consolidated Standards of Reporting Trials
Meta-analysts were having trouble summarizing the literature
Investigators were leaving important information out of their treatment reports

28. CONSORT Requirements
Title: Randomized trial must be stated if the study is an RCT
Methods:
Consort diagram of participant flow
Randomization method
Randomization implementer
Blinding
Defined primary and secondary outcomes
Results:
Baseline demographic data
Number of participants included in analyses
Presentation of effect sizes