Presentation is loading. Please wait.

Presentation is loading. Please wait.

Methods of explanatory analysis for psychological treatment trials workshop Session 3 Analysis of mediation and moderation using instrumental variables.

Similar presentations


Presentation on theme: "Methods of explanatory analysis for psychological treatment trials workshop Session 3 Analysis of mediation and moderation using instrumental variables."— Presentation transcript:

1 Methods of explanatory analysis for psychological treatment trials workshop
Session 3 Analysis of mediation and moderation using instrumental variables Richard Emsley Methodology Research Group Funded by: MRC Methodology Grant G MHRN Methodology Research Group

2 Plan for session 3 Quick review of instrumental variables from Ian’s talk. Why do we use instrumental variables? Where do we find instrumental variables? Examples: PROSPECT mediator example SoCRATES S+A*S model. Designing trials with instruments in mind.

3 Quick review of IVs from Ian’s talk…
Ian has demonstrated how we can use instrumental variable methods to infer a causal effect of treatment in the presence of departures from randomised intervention. This utilises randomisation as the instrumental variable. As we will see, randomisation meets the assumptions required for an IV… But we will also need to consider the situation where we cannot use randomisation as an instrument…

4 Instrumental Variables (IVs)
In a standard regression model, if an explanatory variable is correlated with the error term (known as endogeneity) its coefficient cannot be unbiasedly estimated. An instrumental variable (IV) is a variable that does not appear in the model, is uncorrelated with the error term and is correlated with the endogenous explanatory variable; randomisation, where available, often satisfies this criteria. A two stage least squares (2SLS) procedure can then be applied to estimate the coefficient. At its simplest, the first stage involves using a simple linear regression of the endogenous variable on the instrument and saving the predicted values. In the second stage the outcome is then regressed on the predicted values, with the latter regression coefficient being the required estimate of the coefficient.

5 Some notation Ri – treatment group: the outcome of randomisation (Ri=1 for treatment, 0 for controls). Xi′ = X1i, X2i … Xpi – baseline covariates. Yi – observed outcome. Di – actual treatment received. This is an intermediate outcome that is a putative mediator of the effects of treatment on outcome (either a quantitative measure or binary).

6 Instrumental variables (IV) (from session 1)
Popular in econometrics Simplest idea is: Outcome: Yi = a + b Di + ei Treatment: Di = g + d Ri + fi Allow error ei to be correlated with Di but assume it’s independent of Ri randomisation Ri only affects outcome through its effect on compliance Di Estimation by “two-stage least squares”: E[Yi | Ri] = a + b E[Di | Ri] so first regress Di on Ri to get E[Di | Ri] then regress Yi on E[Di | Ri] NB standard errors not quite correct by this method: general IV uses different standard errors

7 Simple Mediation Idea (from session 2)
dX Mediator β α Treatment Outcomes dY γ The total effect is the sum of the direct effect (γ) and the indirect effect (α*β)

8 Confounded Mediation Diagram
U – the unmeasured confounders dX U Mediator β We have no control over either M or Y (they are both, in fact, outcomes of randomisation). So, there may be unobserved variables, other than treatment (Z), that influence both M and Y. α Treatment Outcomes dY γ If treatment is randomised then assumption of no confounding of treatment and other variables (outcomes) is justified.

9 Confounded Mediation Diagram
dX U U Mediator β α Treatment Outcomes dY γ U If treatment is not randomised then there is likely to be even more unmeasured confounding.

10 Confounded Mediation Diagram
dX U Mediator β α Randomisation Outcomes dY γ Thankfully we’re talking about randomised trials!

11 Linking the two previous sessions: Compliance as a mediator
dX Treatment Received Randomisation Outcomes dY

12 Linking the two previous sessions: Randomisation as an IV
dX Treatment Received Randomisation Outcomes dY By assuming the absence of a direct path from randomisation to outcome, we assume the entire effect of randomisation acts through receipt of treatment. → randomisation is an instrumental variable.

13 Plan for session 3 Quick review of instrumental variables from Ian’s talk. Why do we use instrumental variables? Where do we find instrumental variables? Examples: PROSPECT mediator example SoCRATES S+A*S model. Designing trials with instruments in mind.

14 Why do we use instrumental variables?
All available statistical methods we usually use (for any standard analysis), including: Stratification Regression Matching Standardization require the one unverifiable condition we identified previously: NO UNMEASURED CONFOUNDING

15 Why do we use instrumental variables?
Unlike all other methods, IV methods can be used to consistently estimate causal effects in the presence of unmeasured confounding AND measurement error. SO WE CAN SOLVE THE PROBLEM OF… dX U Mediator β α Randomisation Outcomes dY γ

16 Definition of an instrumental variable
A variable is an instrumental variable Z if: Z has a causal effect on the mediator D; This can be tested in the data. ii. Z affects the outcome Y only through D i.e. there is no direct effect of Z on Y; This is an assumption (sometimes a strong assumption). iii. Z does not share common causes with the outcome Y i.e. there is no confounding for the effect of Z on Y. This is another assumption which randomisation satisfies but other IVs may not.

17 Assumptions for instrumental variables
IV methods require FOUR assumptions The first 3 assumptions are from the definition: The association between instrument and mediator. no direct effect of the instrument on outcome. no unmeasured confounding for the instrument and outcome. There are a wide variety of fourth assumptions and different assumptions result in the estimation of different causal effects: E.g. no interactions, monotonicity (no defiers).

18 Testing assumptions… There are a number of tests we can use for some of these assumptions. Stata has three postestimation commands following ivregress: estat overid estat endogenous estat firststage This final option is perhaps the most useful. It gives an indication of whether the set of instruments strongly predict the mediator – see PROSPECT example later on.

19 Advantages of IVs Can allow for unmeasured confounding;
Can allow for measurement error; Randomisation meets the definition so is an ideal instrument When available. Obviously not in observational studies.

20 Disadvantages of IVs 1. It is impossible to verify that Z is an instrument and using a non instrument introduces additional bias. 2. A weak instrument Z increases the bias over that of ordinary regression. 3. Instruments by themselves are actually insufficient to estimate causal effects and we require additional unverifiable assumptions such as the “no defiers” assumption. 4. Standard IV methods do not cope well with time-varying exposures/mediators…yet. See Hernán and Robins (2006), Epidemiology for further details

21 Assumption trade-off IV methods replace one unverifiable assumption of no unmeasured confounding between the mediator and the outcome by other unverifiable assumptions no unmeasured confounding for the instruments, or no direct effect of the instruments. We need to decide which assumptions are more likely to hold in our mediation analysis. An IV analysis will also increase the precision of our estimates because of allowing for the unmeasured confounding.

22 Also… What about if we want to estimate the direct effect of randomisation in the presence of a potential mediator? dX U Mediator β α Randomisation Outcomes dY γ Clearly we can’t use randomisation as an instrument here…we need another instrument.

23 Plan for session 3 Quick review of instrumental variables from Ian’s talk. Why do we use instrumental variables? Where do we find instrumental variables? Examples: PROSPECT mediator example SoCRATES S+A*S model. Designing trials with instruments in mind.

24 Multiple instruments When we are trying to estimate the direct effect of randomisation we need alternative instruments. Likewise, if we have more than one endogenous variable (multiple mediators), then we need multiple instruments. For IV model identification, we always need to have as many instruments as we have endogenous variables. i.e. if considering two mediators in the model (therapeutic alliance and number of sessions of therapy attended), then we need at least two instrumental variables.

25 Where do we find instruments?
Possibilities for IVs: Randomisation-by-baseline variable interactions. Randomisation involving more than one active treatment – i.e. to interventions specifically targeted at particular intermediate variables/mediators. Randomisation-by-trial (multiple trials). Genetic markers (Mendelian Randomisation) used together with randomisation.

26 Confounded Mediation Diagram
U – the unmeasured confounders dX U Mediator We have no control over either M or Y (they are both, in fact, outcomes of randomisation). So, there may be unobserved variables, other than treatment (R), that influence both M and Y. β α Randomisation Outcomes dY γ If treatment is randomised then assumption of no confounding of treatment and other variables (outcomes) is justified.

27 Mediation Diagram with instruments
U – the unmeasured confounders dX U Randomisation*Covariates Mediator Let’s finally assume that we have measured an important baseline covariate, X. Suppose the effect of R on M is influenced by the value of X. The covariate X is said to be a moderator of the effect of R on M. In addition, X itself is assumed to influence the values of M and to directly influence the values of Y, but there are no covariate by treatment interactions for these components of the model. β α Randomisation Outcomes dY γ Covariates

28 Multiple Instruments Here, treatment by covariates interactions represent instrumental variables. Assumptions: The interactions are significant in the first stage regression (individually and joint F-test). The only effect of the interactions on outcome is through the mediator, and not a direct effect. This is a very strong assumption No other unmeasured confounders between the interactions and outcome.

29 Summary so far… The analysis of mediation is more complex than it first seems because of potential unmeasured confounding (mediators are endogenous). We use moderators of the relationship between randomisation and the mediator (i.e. the baseline by randomisation interactions) as instruments. The analysis of mediation by instrumental variables requires additional assumptions. Primarily, that these covariates are not moderators of the randomisation on outcome relationship (no direct effect). We illustrate these points on two examples now…

30 Plan for session 3 Quick review of instrumental variables from Ian’s talk. Why do we use instrumental variables? Where do we find instrumental variables? Examples: PROSPECT mediator example SoCRATES S+A*S model. Designing trials with instruments in mind.

31 Example: PROSPECT PROSPECT (Prevention of Suicide in Primary Care Elderly: Collaborative Trial) was a multi-site prospective, randomised trial designed to evaluate the impact of a primary care-based intervention on reducing major risk factors (including depression) for suicide in elderly depressed primary care patients. The two conditions were either: (a) an intervention based on treatment guidelines tailored for the elderly with care management, (b) treatment as usual. An intermediate outcome in the PROSPECT trial was whether the trial participant adhered to antidepressant medication during the period following allocation of the intervention. The question here is whether changes in medication adherence following the intervention might explain some or all of the observed (ITT) effects on clinical outcome. See Bruce et al, JAMA (2004); Ten Have et al, Biometrics (2007); Bellamy et al, Clinical Trials (2007); Lynch et al, Health Services and Outcome Research Methodology (2008). Thanks to Tom Ten Have for use of the data.

32 Example: PROSPECT - question of interest
Randomisation*Covariates Antidepressant Use Randomisation Depression Score Covariates

33 Example: PROSPECT - summary stats
Site 1 Site 2 Site 3 Control N=53 Intervention N=53 Control N=57 Intervention N=54 Control N=42 Intervention N=38 Baseline characteristics: number (%) Antidepressant Use 22 (41.5) 18 (34.0) 25 (43.9) 25 (46.3) 25 (59.5) 21 (55.3) Previous medication 27 (50.9) 24 (45.3) 28 (51.9) 29 (69.1) 20 (52.6) Suicidal ideation 9 (17.0) 13 (24.5) 12 (21.1) 18 (33.3) 13 (31.0) 16 (42.1) Post-randomisation adherence to antidepressant medication: number (%) Adherence 20 (37.7) 44 (83.0) 19 (33.3) 45 (83.3) 30 (71.4) 34 (89.5) Hamilton Depression Rating Scores (HRDS): mean (SD) Baseline HDRS 16.5 (5.3) 18.1 (6.2) 17.3 (5.3) 19.9 (6.4) 18.6 (6.3) 18.7 (5.9) 4 month HDRS 13.4 (8.1) 12.0 (7.8) 14.1 (8.6) 12.1 (7.3) 13.0 (8.5) 10.0 (6.9)

34 PROSPECT data – Stata describe
Contains data from P:\SMinMR paper\Prospect.dta obs: vars: Sep :01 size: ,196 (99.9% of memory free) storage display value variable name type format label variable label cad double %10.0g Anti-depressant use at baseline visit hdrs double %10.0g Hamilton depression score at baseline visit ssix double %10.0g Suicide ideation at baseline visit scr double %10.0g Past medication use at baseline visit hdrs double %10.0g Hamilton depression score at 4 month visit site double %10.0g Location of practices interven double %10.0g Randomized assignment to intervention Amedx double %10.0g Adherence to prescribed anti-depressant medication

35 PROSPECT data – Stata ivregress
. xi: ivregress 2sls hdrs4 hdrs0 cad1 ssix01 scr01 i.site i.interven (amedx = i.interven*hdrs0 i.interven*cad1 i.interven*ssix01 i.interven*scr01 i.interven*i.site), first First-stage regressions Number of obs = F( 13, ) = Prob > F = R-squared = Adj R-squared = Root MSE = amedx | Coef. Std. Err t P>|t| [95% Conf. Interval] hdrs0 | cad1 | ssix01 | scr01 | _Isite_2 | _Isite_3 | _Iinterven_1 | _IintXhdrs~1 | _IintXcad1_1 | _IintXssix~1 | _IintXscr0~1 | _IintXsit_~2 | _IintXsit_~3 | _cons |

36 PROSPECT data – Stata ivregress
Instrumental variables (2SLS) regression Number of obs = Wald chi2(8) = Prob > chi2 = R-squared = Root MSE = hdrs4 | Coef. Std. Err z P>|z| [95% Conf. Interval] amedx | hdrs0 | cad1 | ssix01 | scr01 | _Isite_2 | _Isite_3 | _Iinterven_1 | _cons | Instrumented: amedx Instruments: hdrs0 cad1 ssix01 scr01 _Isite_2 _Isite_3 _Iinterven_1 _IintXhdrs0_1 _IintXcad1_1 _IintXssix0_1 _IintXscr01_1 _IintXsit_1_2 _IintXsit_1_3

37 Example: PROSPECT - results
Using all baseline variables as covariates in an ANCOVA. ITT effect: (0.82) Small but statistically significant effect Direct effect Indirect effect γ (s.e.) β (s.e.) Analytical method Standard regression (0.93) (1.09) (Baron & Kenny)

38 Example: PROSPECT - results
Direct effect Indirect effect γ (s.e.) β (s.e.) Analytical method IV (ivreg) (1.35) (2.71) IV (treatreg - ml) (1.27) (2.49) G-estimation* (1.27) (2.34) Conclusion Allowing for hidden confounding appears to have had little effect, except to increase the SE of the estimate. *From Ten Have et al, Biometrics (2007)

39 PROSPECT data – ivregress postestimation
. estat firststage First-stage regressions Number of obs = F( 13, ) = Prob > F = R-squared = Adj R-squared = Root MSE = amedx | Coef. Std. Err t P>|t| [95% Conf. Interval] hdrs0 | cad1 | ssix01 | scr01 | _Isite_2 | _Isite_3 | _Iinterven_1 | _IintXhdrs~1 | _IintXcad1_1 | _IintXssix~1 | _IintXscr0~1 | _IintXsit_~2 | _IintXsit_~3 | _cons |

40 PROSPECT data – ivregress postestimation
(no endogenous regressors) ( 1) _IintXhdrs0_1 = 0 ( 2) _IintXcad1_1 = 0 ( 3) _IintXssix0_1 = 0 ( 4) _IintXscr01_1 = 0 ( 5) _IintXsit_1_2 = 0 ( 6) _IintXsit_1_3 = 0 F( 6, 282) = Prob > F = First-stage regression summary statistics | Adjusted Partial Variable | R-sq R-sq R-sq F(6,282) Prob > F amedx | Minimum eigenvalue statistic = Critical Values # of endogenous regressors: 1 Ho: Instruments are weak # of excluded instruments: 6 | 5% % % % 2SLS relative bias | | 10% % % % 2SLS Size of nominal 5% Wald test | LIML Size of nominal 5% Wald test |

41 Instrumental Variables in SPSS
Analyse – Regression – 2-stage Least Squares Generate interactions as additional variables using compute

42 Instrumental Variables in SPSS
Outcome Covariates and endogenous variable (mediator) Covariates and instruments

43 Example: the SoCRATES trial
SoCRATES was a multi-centre RCT designed to evaluate the effects of cognitive behaviour therapy (CBT) and supportive counselling (SC) on the outcomes of an early episode of schizophrenia. 201 participants were allocated to one of three groups: Control: Treatment as Usual (TAU) Treatment: TAU plus psychological intervention, either CBT + TAU or SC + TAU The two treatment groups are combined in our analyses Outcome: psychotic symptoms score (PANSS) at 18 months

44 Example: SoCRATES - summary stats
Centre 1 - Liv Centre 2 - Man Centre 3 - Nott Mean (SD) Control N=39 Treated N=29 Control N=35 Treated N=49 Control N=26 Treated N=23 Baseline PANSS 80.0 (12.36) 77.7 (13.93) 97.9 (16.6) 100.5 (16.3) 84.9 (14.91) 83.4 (10.84) 18 month PANSS 69.5 (13.55) 50.2 (13.48) 73.2 (22.4) 74.4 (20.00) 54.5 (10.07) 49.1 (7.25) CALPAS - 5.73 (0.81) 5.07 (0.88) 5.15 (1.47) Sessions 18.14 (3.60) 16.16 (4.58) 13.87 (4.95) High Alliance: N(%) 23 (79.3) 30 (61.2) 13 (56.5) # of observed 18m PANSS 23 25 39 21 22 Lewis et al, BJP (2002); Tarrier et al, BJP (2004); Dunn & Bentall, Stats in Medicine (2007); Emsley, White and Dunn, Stats Methods in Medical Research (2009).

45 Confounded Dose-Response
dX U Sessions Attended β α Randomisation Psychotic Symptoms dY Are the effects of Randomisation on Sessions (α) and, more interestingly, the effects of Sessions on Outcome (β), influenced by the strength of the therapeutic alliance?

46 E[Yi(1)-Yi(0)| Xi, Di(1)=s, Di(0)=0 & Ai=a] =
The S + A*S model We want to estimate the joint effects of the strength of the therapeutic alliance as measured by CALPAS (A) and number of sessions attended (S). We postulate a structural model as follows: E[Yi(1)-Yi(0)| Xi, Di(1)=s, Di(0)=0 & Ai=a] = βs*s + βsa*s*(a-7) No sessions implies no treatment effect. The effect of alliance is multiplicative so we only have an interaction effect of alliance – no sessions = no alliance. Dunn and Bentall, SiM (2007)

47 SoCRATES analysis results
Method βs (se) βsa (se) Instrumental variables (0.70) (0.48) Standard regression (B&K) (0.22) (0.11) Note: A has been rescaled so that maximum=0. When A=0 (i.e. maximum alliance) the slope for effect of Sessions is -2.40 When A=-7 (i.e. minimum alliance) the slope is *1.28 = +6.56 This suggests that when alliance is very poor attending more sessions makes the outcome worse!

48 SoCRATES – S + A*S using regress
. regress pant18 sessions s_a pantot logdup c1 c2 yearsed Source | SS df MS Number of obs = F( 7, 145) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = pant18 | Coef. Std. Err t P>|t| [95% Conf. Interval] sessions | s_a | pantot | logdup | c1 | c2 | yearsed | _cons |

49 SoCRATES – S + A*S using ivregress
. ivregress 2sls pant18 pantot logdup c1 c2 yearsed (sessions s_a = group lgp c1gp c2gp yrgp pgp) First-stage regressions Number of obs = F( 11, ) = Prob > F = R-squared = Adj R-squared = Root MSE = sessions | Coef. Std. Err t P>|t| [95% Conf. Interval] pantot | e logdup | e c1 | e c2 | e yearsed | e group | lgp | c1gp | c2gp | yrgp | pgp | _cons | e Model for sessions

50 SoCRATES – S + A*S using ivregress
Number of obs = F( 11, ) = Prob > F = R-squared = Adj R-squared = Root MSE = s_a | Coef. Std. Err t P>|t| [95% Conf. Interval] pantot | e logdup | e c1 | e c2 | e yearsed | e group | lgp | c1gp | c2gp | yrgp | pgp | _cons | e Model for sessions*alliance

51 SoCRATES – S + A*S using ivregress
Instrumental variables (2SLS) regression Number of obs = Wald chi2(7) = Prob > chi2 = R-squared = Root MSE = pant18 | Coef. Std. Err z P>|z| [95% Conf. Interval] sessions | s_a | pantot | logdup | c1 | c2 | yearsed | _cons | Instrumented: sessions s_a Instruments: pantot logdup c1 c2 yearsed group lgp c1gp c2gp yrgp pgp

52 Plan for session 3 Quick review of instrumental variables from Ian’s talk. Why do we use instrumental variables? Where do we find instrumental variables? Examples: PROSPECT mediator example SoCRATES S+A*S model. Designing trials with instruments in mind.

53 Instrumental Variables in observational studies
There are numerous examples of instruments in the absence of randomisation: Access to health care Distance to hospital Genes (known as Mendelian randomisation) Proxy measures of genes (product intolerance) Physician’s preference (ask, or use proportion of patients on treatment)

54 Designing trials with IVs in mind
Thinking back to some of the possibilities for IVs we introduced earlier with design considerations: Randomisation-by-baseline variable interactions. Can we measure any extra baseline variables? Randomisation involving more than one active treatment – i.e. to interventions specifically targeted at particular intermediate variables/mediators. More complicated designs/parallel trials Randomisation-by-trial (multiple trials). Meta-regression approaches (new MRC grant) Genetic markers (Mendelian Randomisation) used together with randomisation. Need to measure genotype in patients

55 Example: Series of parallel trials
Mediator 1 Randomisation 1 Common Outcome Trial 1 Mediator 2 Randomisation 2 Common Outcome Trial 2 Mediator 3 Randomisation 3 Common Outcome Trial 3

56 Example: measuring additional variables
Putative mediator is a measure of the therapist/patient interaction or relationship e.g. Measure of patient’s interaction with other individuals: Care coordinator, family members, etc. e.g. Patient characteristics which could influence ability to form alliance: personality disorders, etc. Similar Baseline measures Therapeutic Alliance Randomisation Outcomes

57 Short small group discussion
We will work in small groups again. We are thinking about designing psychological treatment trials in order to answer some of the explanatory questions discussed in this session? When considering the following potential mediators: How would we accurately measure the mediator? What additional baseline variables might we be able to collect which would help in the causal/IV analysis? What problems could you foresee in the collection of this information? How might you justify the need to collect this information to funders of the trials who would prefer to keep it “large and simple”?

58 Potential mediators for discussion
What are the participant’s beliefs? Does psychotherapy change attributions (beliefs), which, in turn, lead to better outcome? What is the concomitant medication? Does psychotherapy improve compliance with medication which, in turn, leads to better outcome? What is the concomitant substance abuse? Does psychotherapy reduce substance use, which in turn leads to improvements in psychotic symptoms?

59 References – Mediation & Effect Moderation in Psychological Treatment Trials
Methodology for IV methods with mediation: Emsley RA, Dunn G & White IR (2009). Mediation and moderation of treatment effects in randomised trials of complex interventions. Statistical Methods in Medical Research. In press (available online). Maracy M & Dunn G (2009). Estimating dose-response effects in psychological treatment trials: the role of instrumental variables. Statistical Methods in Medical Research. In press (available online). Dunn G & Bentall R (2007). Modelling treatment-effect heterogeneity in randomized controlled trials of complex interventions (psychological treatments). Statistics in Medicine 26, Website with downloads:

60 Some Further Reading Ten Have TR, Joffe MM, Lynch KG, Brown GK, Maisto SA & Beck AT (2007). Causal mediation analyses with rank preserving models. Biometrics 63, Gallop R, Small DS, Lin JY, Elliot MR, Joffe MM & Ten Have TR (2009). Mediation analysis with principal stratification. Statistics in Medicine 28, Bellamy SL, Lin JY & Ten Have TR (2007). An introduction to causal modelling in clinical trials. Clinical Trials 4, Lynch K, Cary M, Gallop R, Ten Have TR (2008). Causal mediation analyses for randomized trials. Health Services & Outcomes Research Methodology 8, Albert JM (2008). Mediation analysis via potential outcomes models. Statistics in Medicine 27, Jo B (2008). Causal inference in randomized experiments with mediational processes. Psychological Methods 13, Gennetian LA, Morris PA, Bos JM & Bloom HS (2005). Constructing instrumental variables from experimental data to explore how treatments produce effects. In: Bloom HS, editor. Learning More From Social Experiments: Evolving Analytic Approaches. New York: Russell Sage Foundation; pp MacKinnon DP (2008). Introduction to Statistical Mediation Analysis. New York: Taylor & Francis Group.


Download ppt "Methods of explanatory analysis for psychological treatment trials workshop Session 3 Analysis of mediation and moderation using instrumental variables."

Similar presentations


Ads by Google