1. SSL spring meeting Turku, Finland, 2017-04-20
Cross Design Synthesis of data from randomized clinical trials and observational studies
SSL spring meeting Turku, Finland,
PD Dr. Christoph Gerlinger, Dr. Tatsiana Vaitsiakhovich, Dr. Anna Filonenko

2. Agenda Problem Statement Cross design synthesis estimator
Example continuation rate of long acting reversible contraception
Summary
Agenda
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

3. Problem statement: Another view on RCTs
“Drugs are tested by the people who manufacture them, in poorly designed trials, on hopelessly small numbers of weird, unrepresentative patients, and analysed using techniques which are flawed by design, in such a way that they exaggerate the benefits of treatments.”
Ben Goldacre, Bad Pharma
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

4. Problem statement: Evidence sources with different strengths and limitations
Randomized clinical trials
Highly selected subset of the total patient population
Protocol-driven procedures and treatments
High internal validity (indispensable for drug licensing)
But, low external validity
Observational studies / registries
More representative of clinical practice
But, internal validity is limited due to confounding, selection bias, channeling, …
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

5. Idea: Combine the strengths of OS and RCT
Several methods proposed in the literature
Confidence profile method
Network meta-analysis and indirect treatment comparison
Cross-design synthesis
Direct modeling of bias
Bayesian hierarchical methods
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

6. Cross design synthesis
GAO reviewed medical study designs, focusing on:
The medical effectiveness research / effectiveness of medical treatments
Evaluation strategies that avoid the limitations of existing approaches
Development of new strategies that avoid limitations of known approaches
GAO noted that:
It devised a strategy that extends the logic of meta-analysis or quantitative overview, which specifies ways of combining results from similar studies
The new strategy, cross design synthesis, combines results from studies of complementary designs in order to capture the designs' strengths and minimize the studies' weaknesses
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

7. Cross design synthesis
Cross design synthesis is a novel strategy for medical effectiveness research, advancing knowledge on medical treatments based on the results of randomized clinical trials and real life evidence Cross design synthesis combines the results from studies that have different complementary designs
Randomized controlled trials
RCTs
Observational studies
OSs
Cross design synthesis
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

8. Kaizar 2011 paper Framework for cross design synthesis based on
Rubin‘s causal model
Stratification (within and between study designs)
Linear model for the relationship of errors between strata
CAVE! Several typos in the formula in the appendix!
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

9. Kaizar paper - stratification
Study type stratification
Randomized vs. Observational
Reflects differential treatment selection error
Population stratification
RCT inclusion criteria met or not
Reflects sample selection error
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

10. Kaizar paper – estimators
Strata specific estimators Cross design estimator
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

11. Properties of CDS estimator
Variance CDS estimator is unbiased if treatment selection error for the patients in the observational trial, fulfilling the inclusion criteria of the RCT, and the patients, who do not, is constant
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

12. Example - introduction
Data from literature
Indication: Long acting reversible contraceptives (LARC)
Research question: How long do women adhere to the method
Assumption for example: similar adherence for all different LARCs (as the data per product were not in the OS publication)
RCT data only from adults
OS data from all ages
Photos:
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

13. Example – RCT 738 women with LARC Mean age 32.1 range 20-41
3 years follow-up
Data taken from supplemental figure 1 of online publication.
Kaplan-Meier estimates were re-calculated considering dropout for „lost to follow-up“ and „other“ as censored (to mimic OS publication as far as possible)
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

14. Example – OS Prospective cohort study 3203 women used LARC
Age (mean 25.7±5.9)
Lost to follow-up and dropout because of wish to get pregnant considered as censored
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

15. Example – Data and Results
Year 1
Year 2
Year 3
OS age 14-19
82. 1 ( )
68.0 ( )
52.6 ( )
OS age 20-45
86.3
( )
76.2
( )
69.2
( )
RCT age 20-41
90.5 ( )
82.4 ( )
79.9 ( )
𝑪𝑫𝑺 𝒀𝒆𝒂𝒓 𝟑 = −69.2 = 𝟕𝟕.𝟖
95% Confidence Interval:
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

16. Example – Strengths and Limitations
CDS estimator adjusted RCT result for excluded adolescents
No huge impact: -2.1 %-point difference in 3-year continuation rate
but only 12.6% adolescents in OS
CDS estimator based on publications could not adjust for other possible patient selection biases in RCT
E.g., 99.3% caucasian in RCT vs. 45.0% in OS
Would need analyses on matched individual patient data
Even with individual patient data one could not adjust for geographic location
RCT from northern and central Europe
OS from St. Louis, Missouri, USA
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

17. Ongoing research and next steps
New OS publication with 2-year data on single products (not LARCs in general)
Analyze individual RCT data as was done in OS publication for CDS
Next steps
Explore CDS method to extrapolate RCT phase III data to real world results for early benefit assessment
Use CDS method in IMI project
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

18. Summary
Cross design synthesis combines results from studies complementary designs in order to capture the designs' strengths and minimize the studies' weaknesses
Not often used so far: 13 hits in PubMed for "cross design synthesis" (as of )
Overlap of patient population in RCT and OS needed
In practice, individual patient data are needed
Individual patient data from > 3,300 industry studies available at
Bottom line: More methods and evaluation of treatment effect heterogeneity and effect modifiers (ideally in RCTs) is needed for CDS and other methods to combine RCT and OS
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

19. Literature Recommended reading
Kaizar, E. E. (2011), Estimating treatment effect via simple cross design synthesis. Statist. Med., 30: 2986–3009. dx.doi.org/ /sim.4339
Verde P.E., Ohmann C. (2015), Combining randomized and non-randomized evidence in clinical research: a review of methods and applications. Res Synth Methods. Mar;6(1): doi: /jrsm.1122
clinicalstudydatarequest.com
Other
GAO Report available from
Kristina Gemzell-Danielsson, Ilka Schellschmidt, Dan Apter, A randomized, phase II study describing the efficacy, bleeding profile, and safety of two low-dose levonorgestrel-releasing intrauterine contraceptive systems and Mirena, Fertility and Sterility, Volume 97, Issue 3, March 2012, Pages e3
Justin T. Diedrich, Qiuhong Zhao, Tessa Madden, Gina M. Secura, Jeffrey F. Peipert, Three-year continuation of reversible contraception, American Journal of Obstetrics and Gynecology, Volume 213, Issue 5, November 2015, Pages 662.e1-662.e8
Abraham M, Zhao Q, Peipert JF. Young Age, Nulliparity, and Continuation of Long-Acting Reversible Contraceptive Methods. Obstet Gynecol Oct;126(4):823-9.
C. Gerlinger, T. Vaitsiakhovich, and A. Filonenko • Cross design synthesis • Turku Finland

20. Thank you!