1. A statistical roadmap for journey from real-world data to real-world evidence
Yixin Fang, Hongwei Wang, Weili He
AbbVie Inc.
@ 2019 NIC-ASA and ICSA Midwest Chapter Meeting

2. Disclaimer
The comments provided here are solely those of the presenter and are not necessarily reflective of the positions, policies or practices of presenter’s employers.
The support of this presentation was provided by AbbVie.   AbbVie participated in
the review and approval of the content.
Yixin Fang, Hongwei Wang, and Weili He are employees of AbbVie Inc.
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

3. Outline Introduction of RWD and RWE
A statistical roadmap of journey from RWD to RWE
Summary
Reference: This presentation is based on our paper
Fang, Wang, He (2019) “A statistical roadmap for journey from real-world data to real-world evidence”; Therapeutic Innovation & Regulatory Science; Accepted
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

4. Real-world data and real-world evidence
The definitions of real-world data and real-world evidence are from the Framework for FDA’s Real-world Evidence Program released in December 2018:
“Real-World Data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.”
“Real-World Evidence (RWE) is the clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD.”
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

5. Examples of real-world data
Excerpts from the FDA’s RWE Framework:
“Examples of RWD include data derived from electronic health records (EHRs); medical claims and billing data; data from product and disease registries; patient-generated data, including from in-home-use settings; and data gathered from other sources that can inform on health status, such as mobile devices.”
Real-world data have two types of data: (1) data relating to patient health status; and (2) data relating to the delivery of health care
Real-world data have two major sources: (1) research data; and (2) transaction data
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

6. A statistical roadmap from RWD to RWE
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

7. Research objective
Then PROTECT Criteria list five key aspects to consider in the forming of research question and objective, when conducting a real-world study
Randomized controlled clinical trials (RCTs) are the gold standard for evaluating the safety and efficacy of pharmaceutical drugs, but in many cases their costs, duration, limited generalizability, and ethical or technical feasibility make us look for real-world studies as alternatives
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

8. A simple example
Consider a cohort study of a specific population of patients (Population)
All patients are exposed to either a treatment of interest or a treatment being compared at the baseline (Treatment/Exposure)
The treatment/exposure is time fixed and all patients are followed from baseline for a same amount of time (Time)
The response/outcome variable is binary and measured at the end of follow-up period (Response/Outcome)
All the potential confounders are measured at baseline and are time fixed (Confounders)
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

9. Observed data If 𝐸 𝑌 𝐴=1 ≠𝐸 𝑌 𝐴=0 ,
or 𝑖 𝐼 𝐴 𝑖 =1 𝑌 𝑖 / 𝑖 𝐼( 𝐴 𝑖 =1) ≠ 𝑖 𝐼 𝐴 𝑖 =0 𝑌 𝑖 / 𝑖 𝐼( 𝐴 𝑖 =0) ,
then we say there is association between 𝐴 and 𝑌
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

10. Target causal quantity
The target causal quantity may be the average treatment effect (ATE)
𝜃 ∗ = 1 𝑁 𝑖=1 𝑁 𝑌 𝑖 1 − 𝑌 𝑖 0
Due to confounding bias, the unadjusted mean difference is a biased estimator of the target causal quantify 𝜃 ∗ . How can we construct an unbiased estimator of 𝜃 ∗
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

11. Identifiability conditions
If there is confounding bias,
𝐸 𝑌 𝐴=1 −𝐸 𝑌 𝐴=0 ≠𝐸 𝑌 1 −𝐸( 𝑌 0 )
In order to adjust for confounding bias, we need to check the following three identifiability conditions
Consistency: 𝑌 𝑎 =𝑌 if 𝐴=𝑎, for 𝑎=0,1. That means, no multiple versions of treatment, perfect compliance, etc
Positivity: 𝑃 𝐴=𝑎 𝑊=𝑤 >0 for 𝑎=0,1 and any 𝑤 with 𝑃 𝑊=𝑤 >0. That means, at any level 𝑤, there are some treated subjects and some untreated subjects
Conditional exchangeability: 𝑌 𝑎 ⫫𝐴|𝑊=𝑤, for 𝑎=0,1 and any 𝑤. This is also known as the assumption of no-unmeasured confounding
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

12. Estimand Under the above three identifiability conditions
𝜃 ∗ =𝐸 𝑌 1 −𝐸 𝑌 0 =𝐸 𝐸 𝑌 1 𝑊 −𝐸 𝐸 𝑌 0 𝑊
=𝐸 𝐸 𝑌 1 𝑊,𝐴=1 −𝐸 𝐸 𝑌 0 𝑊,𝐴=0 =𝐸[𝐸(𝑌│𝑊,𝐴=1)]−𝐸 𝐸 𝑌 𝑊,𝐴=0 = 𝐸 𝑊 [𝐸 𝑌 𝑊,𝐴=1 −𝐸(𝑌|𝑊,𝐴=0)]
For target causal quantity 𝜃 ∗ , the estimand is
𝜃= 𝐸 𝑊 𝑓 0 𝑊,1 − 𝑓 0 𝑊,0
where 𝑓 0 𝑤,𝑎 =𝐸 𝑌 𝑊=𝑤,𝐴=𝑎 is the regression function of 𝑌 on 𝑊 and 𝐴
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

13. Estimator (MLE)
For estimand 𝜃= 𝐸 𝑊 𝑓 0 𝑊,1 − 𝑓 0 𝑊,0 , where 𝑓 0 𝑤,𝑎 =𝐸 𝑌 𝑊=𝑤,𝐴=𝑎 is the regression function of 𝑌 on 𝑊 and 𝐴
Maximum likelihood estimator (MLE): If we can find an estimator of the regression function, say 𝑓 (𝑤,𝑎), based on some model (parametric model like logistic regression model, predictive model like random forest), we can have the following MLE,
𝜃 𝑀𝐿𝐸 = 1 𝑁 𝑖=1 𝑁 𝑓 𝑊 𝑖 , 1 − 𝑓 𝑊 𝑖 ,0
However, MLE is the “best” (unbiased and minimum-variance) estimator of the regression function 𝑓, which is not of direct interest. Instead, we are directly interested in estimating the estimand 𝜃 and we want to construct the “best” estimator of 𝜃
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

14. Estimator (TMLE)
For estimand 𝜃= 𝐸 𝑊 𝑓 0 𝑊,1 − 𝑓 0 𝑊,0 , where 𝑓 0 𝑤,𝑎 =𝐸 𝑌 𝑊=𝑤,𝐴=𝑎 is the regression function of 𝑌 on 𝑊 and 𝐴
Targeted maximum likelihood estimator (TMLE): If we can find some way to update the original estimator 𝑓 𝑤,𝑎 to new estimator 𝑓 ∗ (𝑤, 𝑎), with the goal to conduct the “best” estimator (unbiased and minimum variance) in estimating the estimand 𝜃, then we have the following TMLE,
𝜃 𝑇𝑀𝐿𝐸 = 1 𝑁 𝑖=1 𝑁 𝑓 ∗ 𝑊 𝑖 , 1 − 𝑓 ∗ 𝑊 𝑖 ,0
Book “Targeted learning: causal inference for observational and experiment data” by van der Laan and Rose (2011)
R package “tmle”
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

15. Statistical roadmap from RWD to RWE (revisit)
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

16. Sensitivity analysis
Recall that in the translation from the target causal quantity to the statistical estimand, which in turn is to be estimated by an estimator, we make three identifiability conditions
However, these conditions, particularly the conditional exchangeability condition (which is equivalent to the assumption of no unmeasured confounding) cannot be tested using the measured data
Therefore, we should conduct sensitivity analysis to evaluate how the estimate would change if the assumption of no unmeasured confounder is violated
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

17. A measurement of real-world evidence (E-value)
One recent development of sensitivity analysis is E-value proposed by VanderWeele and Ding (2017), where “E” stands for “evidence”
P-value gives evidence for association
E-value gives evidence that the association is causation
Simply put, E-value measures the real-world evidence that is derived from the analysis of real-world data. This makes the destination of our journey from RWD to RWE
Reference: VanderWeele and Ding (2017) “Sensitivity Analysis in Observational Research: Introducing the E-Value”. Ann Intern Med
R package “EValue”
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

18. Summary
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

19. References
Fang, Wang, He (2019) “A statistical roadmap for journey from real-world data to real-world evidence”; Therapeutic Innovation & Regulatory Science; Accepted
Franklin and Schneeweiss (2017) “When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials?” Clin Pharmacol Ther. 102(6):
Hernan and Robins (2019) “Causal Inference”. Boca Raton: Chapman & Hall/CRC
VanderWeele and Ding (2017) “Sensitivity Analysis in Observational Research: Introducing the E-Value”; Ann Intern Med.; 167(4):
van der Laan and Rose (2011) “Targeted Learning: Causal Inference for Observational and Experimental Data” Springer
van der Laan and Rose (2018) “Targeted Learning in Data Science: Causal Inference for Complex Longitudinal Studies” Springer
"Roadmap from RWD to RWE" at NIC-ASA and ICSA Midwest Chapter Meeting on 10/25/19

20. Thank You !