1. The Incorporation of Meta-Analysis Results into Evidence-Based Decision Modelling Nicola Cooper, Alex Sutton, Keith Abrams, Paul Lambert, David Jones Department of Epidemiology & Public Health, University of Leicester. CHEBS, Multi-Parameter Evidence Synthesis Workshop, Sheffield, March 2002

2. Where we fit in with Tony’s intro Process – Model relationship between evidence & parameters –Consistency check Uncertainty Panacea – Statistical error –½ Evidence relates to parameters indirectly –Systematic errors –Data quality, publication bias, etc

3. 1) Pooled estimates METHODOLOGIC PRINCIPLE

4. 1) Pooled estimates 2) Distribution METHODOLOGIC PRINCIPLE

5. 1) Pooled estimates 3) Transformation of distribution to transition probability (if required) 2) Distribution (i) time variables: (ii) prob. variables: METHODOLOGIC PRINCIPLE

6. 1) Pooled estimates 3) Transformation of distribution to transition probability (if required) 2) Distribution 4) Application to model (i) time variables: (ii) prob. variables: METHODOLOGIC PRINCIPLE

7. 1)Net Clinical Benefit Approach Warfarin use for atrial fibrillation 2)Simple Economic Decision Model Prophylactic antibiotic use in caesarean section 3)Markov Economic Decision Model Taxane use in advanced breast cancer EXAMPLES

8. Bayesian methods implemented using Markov Chain Monte Carlo simulation within WinBUGS software Random effect meta-analysis models used throughout All prior distributions intended to be ‘vague’ unless otherwise indicated Where uncertainty exists in the value of parameters (i.e. most of them!) they are treated as random variables All analyses (decision model and subsidiary analyses) implemented in one cohesive program MODELLING ISSUES COMMON TO ALL EXAMPLES

9. EXAMPLE 1: NET (CLINICAL) BENEFIT Net Benefit = (Risk level x Risk reduction) – Harm Glasziou, P. P. and Irwig, L. M. An evidence based approach to individualizing treatment. Br.Med.J. 1995; 311:1356-1359.

10. Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin However, there is a risk of a fatal hemorrhage as a result of taking warfarin For whom do the benefits outweigh the risks? RE-ANALYSIS OF WARFARIN FOR NON- RHEUMATIC ATRIAL FIBRILLATION

11. 1)Perform a meta-analysis of the RCTs to estimate the relative risk for benefit of the intervention 2)Use this to check the assumption that RR does not vary with patient risk 3)Check harm (adverse events) is constant across levels of risk (use RCTs and/or data from other sources) & estimate this risk 4)Place benefit & harm on same scale (assessment of QoL following different events) 5)Apply model - need to predict patients risk (identify risk factors and construct multivariate risk prediction equations) METHOD OUTLINE

12. SOURCES OF EVIDENCE Net Benefit = (risk of stroke x relative reduction in risk of stroke) - (risk of fatal bleed x outcome ratio) Multivariate risk equations M-A of RCTs M-A of RCTs/obs studies QoL study

13. Singer,D.E. Overview of the randomized trials to prevent stroke in atrial fibrillation. Ann Epidemiol 1993;3:567-7.

14. EVALUATING THE TRADE-OFF BETWEEN STROKE AND HEMORRHAGE EVENTS IN TERMS OF QOL QoL following a fatal bleed = 0 Data available on QoL of patients following stroke –Glasziou, P. P., Bromwich, S., and Simes, R. J. Quality of life six months after myocardial infarction treated with thrombolytic therapy. The Medical Journal of Australia. 1994; 161532-536 Proportion with index greater than horizontal axis value Time trade-off index

15. EVALUATION OF NET BENEFIT (risk of stroke  relative reduction in risk of stroke) - (risk of fatal bleed  outcome ratio) = Net Benefit Multivariate risk equations Meta-analyses ofRCTs Meta-Analysis of RCTs/obsstudies QoLstudy 0.0020.0040.0060.0080.0100.0120.014 0 50 100 150 200 250 300 risk of bleed per year -2.95-2.90-2.85-2.80-2.75-2.70-2.65 0 2 4 6 8 10 020406080100 0.0 0.1 0.2 0.3 0.4 Outcome ratio Relative risk reduction for strokes taking warfarin(1-RR): 0.23 (0.13 to 0.41) Outcome ratio (1/QoL reduction) Median 3.75 (1.07 to 50), Mean 26.14,indicating the number of strokes that are equivalent to one death Risk of stroke per year e.g. for 1 or 2 clinical risk factors: 6.0% (4.1 to 8.8) Risk of fatal bleed per year taking warfarin: 0.52% (0.27 to 0.84)

17. “TAKE-HOME” POINTS 1 d Net-benefit provides a transparent quantitative framework to weigh up benefits and harms of an intervention d Utilises results from two meta-analyses and allows for correlation induced where studies included in both benefit and harm meta-analyses d Credible interval for net benefit can be constructed allowing for uncertainty in all model parameters

18. Use of prophylactic antibiotics to prevent wound infection following caesarean section EXAMPLE 2: SIMPLE DECISION TREE

19. 1)Cochrane review of 61 RCTs evaluating prophylactic antibiotics use for caesarean section 2)Event data rare: use “Exact” model for RR 3)Meta-regression: Does treatment effect vary with patients’ underlying risk (pc)? ln(RR adjusted ) = ln(RR average )+  [ln(pc) - mean(ln(pc))] 4)Risk of infection without treatment from ‘local’ hospital data (p1) 5)Derive relative risk of treatment effect for ‘local’ hospital (using regression equation with pc=p1) 6)Derive risk of infection if antibiotics introduced to ‘local’ hospital (p2) p2 = p1 * RR adjusted METHOD OUTLINE

20. UNDERLYING BASELINE RISK  =0.24 (-0.28 to 0.81) Local hospital event rate No treatment effect

21. Mean (95% Credible Interval) Posterior distribution Relative Risk, RR adjusted 0.30 (0.21 to 0.40) Prob(wound infection/placebo), p1 0.08 (0.06 to 0.10) Prob(wound infection/antibiotics), p2 0.02 (0.015 to 0.034) RESULTS

22. Mean (95% Credible Interval) Posterior distribution Reduction in cost using antibiotics -£49.53 (-£77.09 to -£26.79) Number of wound infections avoided using antibiotics per 1,000 53.09 (42.12 to 73.37) Between study variance (random effect in M-A),  2 0.30 (0.05 to 0.74) RESULTS (cont.)

23. COST-EFFECTIVENESS PLANE Control dominates Treatment less effective & less costly Treatment dominates Treatment more effective & more costly

24. SENSITIVTY OF PRIORS [1] G amma(0.001,0.001) on  2 [2] Normal(0,1.0 -6 ) truncated at zero on  [3] Uniform(0,20) on 

25. “TAKE-HOME” POINTS 2 d Incorporates M-A into a decision model adjusting for a differential treatment effect with changes in baseline risk d Meta-regression model takes into account the fact that covariate is part of the definition of outcome d Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propogated d Sensitivity of overall results to prior distribution placed on the random effect term in a M-A

26. EXAMPLE 3: USE OF TAXANES FOR 2 ND LINE TREATMENT OF BREAST CANCER Stages 1 & 2 (cycles 1 to 3) Stage 3 (cycles 4 to 7) Stage 4 (cycles 8 to 35) Treatment cycles Post - Treatment cycles

27. 1)Define structure of Markov model 2)Identify evidence used to inform each model parameter using meta-analysis where multiple sources available 3)Transform meta-analysis results, where necessary, into format required for model (e.g. rates into transition probabilities) 4)Informative prior distributions derived from elicited prior beliefs from clinicians 5)Evaluate Markov model METHOD OUTLINE

28. META-ANALYSES

29. TRANSITION PROBABILITIES

30. 1) Pooled estimates 3) Transformation of distribution to transition probability (if required) 2) Distribution 4) Application to model (i) time variables: (ii) prob. variables: METHODOLOGIC PRINCIPLE

31. ELICITATION OF PRIORS e.g. Response Rate Taxane Standard

32. RESPONSE RATE

33. COST-EFFECTIVENESS PLANE

34. “TAKE-HOME” POINTS 3 d Synthesis of evidence, transformation of variables & evaluation of a complex Markov model carried out in a unified framework (facilitating sensitivity analysis) d Provides a framework to incorporate prior beliefs of experts

35. FURTHER ISSUES Handling indirect comparisons correctly E.g. Want to compare A v C but evidence only available on A v B & B v C etc. Avoid breaking randomisation Necessary complexity of model? When to use approaches 1,2,3 above? Use of predictive distributions Necessary when inferences made at ‘unit’ level (e.g. hospital in 2 nd example) rather than ‘population’ level? Incorporation of EVI

37. MODEL SPECIFICATION Warn et al 2002 Stats in Med (in press) Bayesian random effects M-A model specification: ln(RR) Prior distributions: