Presentation on theme: "SESSION 2 Mediation and moderation of treatment effects Andrew Pickles Methods of explanatory analysis for psychological treatment trials workshop Methodology."— Presentation transcript:
SESSION 2 Mediation and moderation of treatment effects Andrew Pickles Methods of explanatory analysis for psychological treatment trials workshop Methodology Research Group Funded by: MRC Methodology Grant G MHRN Methodology Research Group
2 Moderators & Mediators Moderator is a variable that modifies the form or strength of the relation between an independent and a dependent variable. Mediator is a variable that is intermediate in the causal sequence relating an independent variable to a dependent variable.
3 Moderators in RCTs Moderators are baseline characteristics that influence the effect of treatment, or the effect of treatment allocation (on intermediate or final outcomes). They are pre-randomisation effect- modifiers. Examples: sex, age, previous history of mental illness, insight, treatment centre, therapist characteristics, genes etc.
4 Typical local example Creed et al., Psychosomatic Medicine 67:490–499 (2005) Figure 2. SF36 scores by abuse categories at baseline and follow-up (treated patients only)
5 Testing for Moderation A moderator variable is typically a baseline variable (e.g. not-abused, abused) Makes treatment effect greater in one group than another (moderator may or may not have an additional direct effect on outcome). It is a source of treatment effect heterogeneity A classic error is to claim moderation when treatment effect is significant effect in one group and not significant in another. Is simply a recipe for increasing Type I (false positive) error rate
6 Interaction & Synergy Need to show significant interaction with treatment on outcome But on what scale? –Can find that interaction significant on one scale but is not significant if outcome variable is transformed. Choice of scale requires both statistical and clinical considerations. –If outcome binary then usual test is for interaction on the log- odds scale Some argue that main effects on log-odds scale already suggests synergy –e.g. if the base outcome rate is low and the treatment and moderator each increase outcome by 100% then the two together increase the outcome rate not by 200% but by 300% even without an interaction
7 The SoCRATES Trial SoCRATES was a multi-centre RCT designed to evaluate the effects of cognitive behaviour therapy (CBT) and supportive counselling (SC) on the outcomes of an early episode of schizophrenia. Participants were allocated to one of three conditions: Analysed as two conditions Control condition: Treatment as Usual (TAU), Treatment condition: TAU plus psychological, either CBT + TAU or SC + TAU.
8 SoCRATES (contd.) 3 treatment centres: Liverpool, Manchester and Nottinghamshire. Other baseline covariates include logarithm of untreated psychosis and years of education. Outcome (a psychotic symptoms score) was obtained using the Positive and Negative Syndromes Schedule (PANSS). From an ITT analyses of 18 month follow-up data, both psychological treatment groups had a superior outcome in terms of symptoms (as measured using the PANSS) compared to the control group.
9 SoCRATES (contd.) Post-randomization variables that have a potential explanatory role in exploring the therapeutic effects include the total number of sessions of therapy actually attended and the quality or strength of the therapeutic alliance. Therapeutic alliance was measured at the 4th session of therapy, early in the time-course of the intervention, but not too early to assess the development of the relationship between therapist and patient. We use a patient rating of alliance based on the CALPAS (California Therapeutic Alliance Scale). Total CALPAS scores (ranging from 0, indicating low alliance, to 7, indicating high alliance) were used in some of the analyses reported below, but we also use a binary alliance variable (1 if CALPAS score ≥5, otherwise 0)..
10 SoCRATES - Summary Statistics Centre 1 - LivCentre 2 - ManCentre 3 - Nott Mean (SD) Control N=39 Treated N=29 Control N=35 Treated N=49 Control N=26 Treated N=23 Baseline PANSS80.0 (12.36)77.7 (13.93)97.9 (16.6)100.5 (16.3)84.9 (14.91)83.4 (10.84) 18 month PANSS69.5 (13.55)50.2 (13.48)73.2 (22.4)74.4 (20.00)54.5 (10.07)49.1 (7.25) CALPAS-5.73 (0.81)-5.07 (0.88)-5.15 (1.47) Sessions (3.60) (4.58) (4.95) High Alliance: N(%)-23 (79.3)-30 (61.2)-13 (56.5) # of observed 18m PANSS Lewis et al, BJP (2002); Tarrier et al, BJP (2004); Dunn & Bentall, Stats in Medicine (2007).
11 Socrates positive symptoms: basic analysis xi: regress enpstot psubtota rgrp Source | SS df MS Number of obs = F( 2, 222) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval] psubtota | rgrp | _cons |
12 Socrates positive symptoms: including main effects of centre xi: regress enpstot psubtota i.centre rgrp enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval] psubtota | _Icentre_2 | _Icentre_3 | rgrp | _cons | testparm _Icen* ( 1) _Icentre_2 = 0 ( 2) _Icentre_3 = 0 F( 2, 220) = Prob > F =
13 Socrates positive symptoms: treatment effect by centre xi:xi: bysort centre : regress enpstot psubtota rgrp -> centre = enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval] psubtota | rgrp | _cons | > centre = enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval] psubtota | rgrp | _cons | > centre = enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval] psubtota | rgrp | _cons |
15 Mediators in Randomised Clinical Trials (RCTs) Mediators are intermediate outcomes on the causal pathway between allocation to or receipt of treatment and final outcome. By definition, in an RCT, they are measured after randomisation. Treatment effect may be fully or partially explained by a given mediator. Possible for a given mediator to serve the role of surrogate outcome. Possibility of multiple mediators (multiple pathways) and interactions between mediators.
16 Post-randomisation effect modifiers Intermediate outcomes that influence either (a) the effects of treatment/treatment allocation on other intermediate outcomes (mediators) or (b) the effects of the other intermediate outcomes on the final outcome. Candidates: amount of treatment (sessions attended), treatment fidelity, therapeutic alliance. Distinction between these variables and mediators not obvious.
17 Examples Compliance with allocated treatment Does the participant turn up for any therapy? How many sessions does she attend? Fidelity of therapy How close is the therapy to that described in the treatment manual? Is it a cognitive-behavioural intervention, for example, or merely emotional support? Quality of the therapeutic relationship What is the strength of the therapeutic alliance?
18 Examples (cont.) What is the concomitant medication? Does psychotherapy improve compliance with medication which, in turn, leads to better outcome? What is the direct effect of psychotherapy? What is the concomitant substance abuse? Does psychotherapy reduce cannabis use, which in turn leads to improvements in psychotic symptoms? What are the participant’s beliefs? Does psychotherapy change attributions (beliefs), which, in turn, lead to better outcome? How much of the treatment effect is explained by changes in attributions?
19 The Mediation Industry Baron RM & Kenny DA (1986). The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology 51, As of 16 th September 2009: 12,292 citations! Assumptions are very rarely stated, let alone their validity discussed. One suspects that the majority of investigators are oblivious of the implications.
20 Number of sessions Mediator M Psychotic Symptoms Dependent Y Randomised to Psych treatment Independent X e3e3 e2e2 Regression eqns used to assess mediation Y=d 1 +cX+e 1 Y=d 2 +c’X+bM+e 2 M=d 3 +aX+e 3 total effect=c mediated effect= ab or (c-c’) (in simple linear models these should be equal if estimated on same sample) a b c’ A Naïve Look at mediation: the B&K framework
21 Testing for Mediation Estimate of mediated effect = Confidence interval +/- 1.96*se ab Estimate of se ab = sqrt( se b 2 + se a 2 ) Bootstrap resampling better (allows for asymmetry) Test of mediation (1) if 0 within CI (2) z-test for /se ab
22 Baron & Kenny Steps: naïve mediation Effect of X on Y (c) must be significant Effect of X on M (a) must be significant Effect of M on X (b) must be significant When controlling for M, the direct effect of X on Y (c’) must be non-significant
24 naïve mediation xi:regress enpstot nosess rgrp psubtota i.centre enpstot | Coef. Std. Err. t P>|t| [95% Conf. Interval] nosess | rgrp | psubtota | _Icentre_2 | _Icentre_3 | _cons | A=13.80 (0.59), B=0.042 (0.08): A times B =0.58 (1.10) Sobel estimate of standard error sqrt( * * )=1.10
25 Stata code for naïve mediation: bootstrap 1 global model1 “nosess rgrp psubtota i.centre" global model2 “enpstota nosess rgrp psubtota i.centre" program mediate, rclass version 8 xi:regress $model1 matrix a=e(b) xi:regress $model2 matrix b=e(b) return scalar mediate=a[1,1]*b[1,1] end bootstrap mediate product=r(mediate), reps(100) dots
26 Stata code for naïve mediation: bootstrap 2 bootstrap mediate product=r(mediate), reps(100) dots command: mediate statistic: product = r(mediate) Bootstrap statistics Number of obs = 213 Replications = Variable | Reps Observed Bias Std. Err. [95% Conf. Interval] product | (N) | (P) | (BC) Note: N = normal P = percentile BC = bias-corrected
27 Mediation and measurement error: mediation or direct and indirect effects in SEM (Mplus) Testing for and estimating mediation can be susceptible to measurement error bias
28 Direct and Indirect: Longitudinal and sleeper effects y1y2y3y4 abc y1 directly influences y2 through path a y1 only indirectly influences y3 through y2 on paths a and b In a longitudinal study if y1 influences y3 directly (i.e. not through y2) this is a ‘sleeper effect’ This structure of restricting effects to those from the previous occasion is known as first order autorgression (AR1)
29 Longitudinal Ability Data % correct at ages 6,7, 9 and 11 STANDARD DEVIATIONS CORRELATION MATRIX
30 AR1 Model: ability1.inp TITLE: Ability autoregressive model DATA: FILE IS D:\courses\mplus\ability.dat; TYPE IS CORRELATION STDEVIATIONS; NOBSERVATIONS=204; VARIABLE: NAMES ARE y1-y4; USEVARIABLES ARE y1-y4; MODEL: y2 on y1; y3 on y2; y4 on y3; OUTPUT: SAMPSTAT STANDARDIZED RESIDUAL;
31 Indirect effects: Ability1b.inp TITLE: Ability latent autoregressive model DATA: FILE IS D:\courses\mplus\ability.dat; TYPE IS CORRELATION STDEVIATIONS; NOBSERVATIONS=204; VARIABLE: NAMES ARE y1-y4; USEVARIABLES ARE y1-y4; MODEL: y2 on y1; y3 on y2; y4 on y3; MODEL INDIRECT: y4 IND y1; y3 IND y1; OUTPUT:STANDARDIZED; CINTERVAL;
32 AR1 Model Output-1 Effects from Y1 to Y4 Total Total indirect Specific indirect Y4 Y3 Y2 Y Effects from Y1 to Y3 Total Total indirect Specific indirect Y3 Y2 Y
33 Autoregressive Output-1 Chi-Square Test of Model Fit Value ! This fits Degrees of Freedom 3 ! Very badly P-Value ESTIMATED MODEL AND RESIDUALS (OBSERVED - ESTIMATED) Model Estimated Covariances/Correlations/Residual Correlations Y2 Y3 Y4 Y1 ________ ________ ________ ________ Y Y Y Y Residuals for Covariances/Correlations/Residual Correlations Y2 Y3 Y4 Y1 ________ ________ ________ ________ Y Y Y Y
34 Autoregressive Output-2 TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND DIRECT EFFECTS Estimates S.E. Est./S.E. Std StdYX Effects from Y1 to Y4 Total Total indirect Specific indirect Y4 Y3 Y2 Y Effects from Y1 to Y3 Total Total indirect Specific indirect Y3 Y2 Y
35 Simplex Model f1f2f3f4 y1y2y3y4 Age 6 Age 7 Age 9 Age 11 V’s measured with error Autoregressive F’s Curiously, middle part of model is identified without restrictions, but the whole model is not identified without some restrictive assumptions e.g. measurement error and reliability constant with age
36 Simplex Model: ability2.inp TITLE: Ability latent autoregressive model DATA: FILE IS D:\courses\mplus\ability.dat; TYPE IS STDEVIATIONS CORRELATION; NOBSERVATIONS=204; VARIABLE: NAMES ARE y1-y4; USEVARIABLES ARE y1-y4; MODEL: f1 by y1 (1); f2 by y2 (1); f3 by y3 (1); f4 by y4 (1); y1 y2 y3 y4 (2); f2 on f1; f3 on f2; f4 on f3; MODEL INDIRECT: f3 IND f1; f4 IND f1; OUTPUT: STANDARDIZED;
37 Simplex Model: ability2.out TESTS OF MODEL FIT Chi-Square Test of Model Fit Value Degrees of Freedom 2 P-Value TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND DIRECT EFFECTS Estimates S.E. Est./S.E. Std StdYX Effects from F1 to F3 Total Total indirect Specific indirect F3 F2 F Effects from F1 to F4 Total Total indirect
38 Simplex Model: conclusion Conclusion. In the presence of measurement error in the mediator the mediated effect is underestimated (attenuated) and the residual “direct” effect over-estimated. With multiple predictors (mediators) measurement error can result in decreased, increased and quite spurious effects being estimated. But still ignores possible confounding – to be addressed this afternoon