RCTs and instrumental variables Anna Vignoles University of Cambridge

Why do you need an IV in an RCT? RCTs randomize the allocation of the treatment But not everyone complies People used to analyse the data “as treated” – Treatment on the treated ignoring the fact that some people who were randomised into the treatment did not participate This is generally a bad solution because those who choose to participate are not the same as those who don’t!

Why do you need an IV in an RCT? Nowadays the preferred analytical solution is Intention to Treat (ITT approaches) – Difference in outcomes between those who are randomised into the treatment and those who are not But ITT tells you impact of offering the programme We would still like to know the effect of the treatment on the treated But the treated are not a random subset….

What about an IV solution? IV often used post hoc to evaluate a programme – Maimonides rule in Israel, Victor Lavy Can be used ex ante – Design an IV into an evaluation – Design an IV into a RCT In medical literature use of IV in a trial is called contamination adjusted intention to treat

Why use an IV in a RCT? Computing the ITT – Straight difference in average outcomes between the group to whom you offered treatment, and the group to whom you did not offer treatment Computing the Effect of Treatment on the Treated (TOT) – Use whether or not the person was randomised into the intervention (Z) to predict whether or not the individual actually participated in the intervention (D)

Instrumental Variables: a refresher Y 1i is the value of the outcome if the treatment is received by individual i Y 0i is the value of the outcome if the treatment is not received by individual i D i = 1 if treatment is received by individual i D i = 0 if treatment is not received by individual i X i denotes the set of observed characteristics before the intervention/treatment for individual i

Instrumental Variables: a refresher D i is composed of two parts, one that is correlated with u (endogenous part) and one that is independent of the error term (exogenous part) IV uses an additional variable(s) Z (called an instrumental variable, to isolate that part of D that is correlated with the error term In this case Z is the randomisation process

Instrumental Variables: a refresher For a valid instrument the following must be true: – corr (Z i,D i ) > 0Instrument is relevant – E(u i |Z i, X i )= E(u i |X i )= 0Instrument effects D, but not Y directly (only through its impact on D) The instrument must predict D The instrument must also only effect Y through its impact on D (untestable assumption) IV is estimated by 2 Stage Least Squares

Problems with IV If instruments are weakly correlated with the endogenous variable, the instruments are said to be weak When using weak instruments the IV 2SLS estimator is biased even in large samples In small samples IV estimates are biased anyway Finite sample bias will lessen as sample size increases In this case clear strong instrument….

Advantages and disadvantages Essentially adjusts estimate for degree of non compliance Information on non compliance can be revealing in itself to understand the impact of the intervention Non compliance may be difficult to measure in practice – incomplete or partial compliance Assumes that if the non compliers had received the treatment the effect for them would have been the same as for the compliers Assumptions behind ITT – the effect of the treatment is averaged over those who actually receive it and those who do not

Some examples Vitamin A supplementation in malnourished children reduced mortality by 41% using ITT Supplements was found to reduce mortality by two thirds (72%) using CA ITT – Sommer and Zeger 1991

References Angrist, J. D. and A. Krueger (2001). “Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments”, Journal of Economic Perspectives, 15(4). Heckman, James J. "Randomization as an instrumental variable." (1995). Imbens, G. W. and J. D. Angrist, (1994). “Identification and Estimation of Local Average Treatment Effects.” Econometrica, 62(2). Sommer A, Zeger SL. On estimating efficacy from clinical trials. Stat Med1991;10:45-52 Sussman, Jeremy B., and Rodney A. Hayward. "An IV for the RCT: using instrumental variables to adjust for treatment contamination in randomised controlled trials." Bmj 340 (2010): c2073.