Reference based sensitivity analysis for clinical trials with missing data via multiple imputation Suzie Cro 1,2, Mike Kenward 2, James Carpenter 1,2 1 MRC Clinical Trials Unit at UCL 2 The London School of Hygiene and Tropical Medicine
Outline Reference based multiple imputation; asthma trial Motivation for evaluation Information anchoring principle Investigation of Rubin’s multiple imputation variance estimator Conclusions
Example – asthma trial Placebo vs. Budesonide for patients with chronic asthma Forced Expiratory Volume in 1 second (FEV 1 ) recorded at baseline, 2, 4, 8 and 12 weeks Primary outcome: mean treatment group difference at 12 weeks Only 37/90 Placebo and 71/90 Budesonide completed Busse et al. (1998)
Example – asthma trial Any analysis must make an untestable assumption about the unobserved data Wrong assumption biased treatment estimate Primary analysis – Missing-at-Random (MAR) A set of analyses where the missing data is handled in different ways as compared to the primary analysis should be undertaken
Example – asthma trial - MAR Placebo MAR means Active MAR means Time (weeks)
Example – asthma trial - MAR Observed FEV 1 Placebo MAR means Active MAR means Time (weeks)
Example – asthma trial - MAR Observed FEV 1 Placebo MAR means Active MAR means Time (weeks)
Example – asthma trial - MAR Imputed FEV 1 Observed FEV 1 Placebo MAR means Active MAR means Time (weeks)
Example – asthma trial – Jump to reference Imputed FEV 1 Observed FEV 1 Placebo MAR means Active MAR means Time (weeks)
Reference based sensitivity analysis ‘Fill in’ the missing data under reference based assumption multiple times Analyse each imputed data set using the primary design based analysis model Get one overall treatment effect and estimate of variance using Rubin’s rules Carpenter, Roger and Kenward (2013)
Reference based sensitivity analysis ‘Fill in’ the missing data under reference based assumption multiple times Analyse each imputed data set using the primary design based analysis model Get one overall treatment effect and estimate of variance using Rubin’s rules Carpenter, Roger and Kenward (2013)
Motivation for evaluation Rubin's variance formula requires the imputation and analysis models to be congenial - the imputation and analysis model must have the same content and structure Here the analysis model makes different assumptions to the imputation model, violating the usual justification!
Motivation for evaluation 1.What properties do we require of our variance estimate in the reference based sensitivity analysis setting? 2.Does Rubin’s variance estimator provide an appropriate estimate of variance?
Information anchoring A loss of information is inevitable with missing data The extent of information loss in primary analysis (MAR) is intrinsically linked to the analysis model A natural principle for the treatment estimator variance is to keep the information loss due to missing data constant or anchored across primary and all sensitivity analysis Sensitivity analyses should not inject information ‘by the back door’
Information anchoring Variance estimate Full data; no deviations occur Full data; deviation data follow the specific reference assumption Deviation data are missing and multiply imputed - MAR For the design based analysis model let,
Information anchoring Variance estimate Full data; no deviations occur Full data; deviation data follow the specific reference assumption Deviation data are missing and multiply imputed - MAR For the design based analysis model let,
Information anchoring Variance estimate Full data; no deviations occur Full data; deviation data follow the specific reference assumption Deviation data are missing; analysis under MAR For the design based analysis model let,
Information anchoring Variance estimate Full data; no deviations occur Full data; deviation data follow the specific reference assumption Deviation data are missing; analysis under MAR For the design based analysis model let,
Information anchoring Variance estimate Full data; no deviations occur Full data; deviation data follow the specific reference assumption Deviation data are missing; analysis under MAR For the design based analysis model let,
Rubin’s variance estimate Proposition: In all settings where,
Rubin’s variance estimate
Software Mimix implements the reference based MI procedures in Stata Download in Stata using ssc install mimix SAS Macro’s available at
Conclusions The loss of information due to missing data across primary and design based sensitivity analysis should be constant Rubin’s variance estimate anchors the loss of information in the primary analysis to a very good approximation
Conclusions The loss of information due to missing data across primary and design based sensitivity analysis should be constant Rubin’s variance estimate anchors the loss of information in the primary analysis to a very good approximation Rubin’s rules provide an appropriate estimate of variance when using reference based multiple imputation for sensitivity analysis
Carpenter JR, Roger JH, Kenward MG, Analysis of Longitudinal Trials with protocol deviation: a framework for relevant accessible assumptions and inference via multiple imputation, Journal of Biopharmaceutical Statistics, 23: , Carpenter JR, Roger JH, Cro S, Kenward MG, Response to comments by Seaman et al. on ‘Analysis of Longitudinal Trials with protocol deviation: a framework for relevant accessible assumptions and inference via multiple imputation’, Journal of Biopharmaceutical Statistics, 24: , Cro S, Morris TP, Kenward MG, Carpenter JR, Reference-based sensitivity analysis via multiple imputation for longitudinal trials with protocol deviation, Stata Journal, 16:2: , Rubin DB, Multiple imputation for nonresponse in surveys, Wiley series in probability and mathematical statistics, USA: John Wiley & Sons, 1987.