1. Introduction to Bayesian Statistical Software: WinBUGS 1.4.3 Beth Devine, PharmD, MBA, PhD Rafael Alfonso, MD, PhC Evidence Synthesis 9/22/2011 2:30 pm Supported by the Institute of Translational Health Sciences, Grant NIH 3 UL1 RR 025014-04S2 and the UW CHASE Alliance

2. Comparative Effectiveness of Biologic Therapies in Rheumatoid Arthritis (RA): An Indirect Treatment Comparisons Approach Beth Devine, PharmD, MBA, PhD Rafael Alfonso-Cristancho, MD, MS Sean Sullivan, BSPharm, PhD Pharmaceutical Outcomes Research & Policy Program University of Washington Pharmacotherapy 2011;31:39–51

3. WinBUGS – Model Syntax

4. WinBUGS – Load Data

5. WinBUGS – Model Compiled

6. WinBUGS – Model Initialized

7. WinBUGS – Update (Burn-in)

8. WinBUGS – Check Convergence

9. WinBUGS – Obtaining Posterior Inference

10. WinBUGS – Viewing Summary Statistics

11. WinBUGS – Interpreting Summary Statistics Check start and sample columns – 10,000 to 30,000 Rename your parameters Assess accuracy of posterior estimates by calculating Monte Carlo error for each parameter: Rule of thumb: MC error should be < 5% of sample standard deviation Exponentiate median log odds to odds ratios

12. Now Introducing our Practice Dataset

13. Advantage of Bayesian analysis in ITC/ MTC is that it allows calculation of the probability of which treatment is best http://www.mrc-bsu.cam.ac.uk/bugs/http://www.mrc-bsu.cam.ac.uk/bugs/ or Lunn, Thomas, Best, Speigelhalter. Stat Comput 2000;10:325-37

14. Outcome Measures How is your outcome of interest measured? –Binary (e.g. dead or alive) –Continuous (e.g blood pressure) –Categorical/ordinal (e.g. severity scale) Binary outcomes most common –We will consider here Continuous –Similar approach to binary Ordinal –More complex and more rare

15. Binary Outcome Measures Binary outcome data from a comparative study can be expressed in a 2 x 2 table Three common outcome measures: –Odds ratios, risk ratios, risk differences Failure/DeadSuccess/Alive New TreatmentAB ControlCD RCT

16. Fixed Effects Model Statistical homogeneity Formally assume: Y i = Normal(d,V i ) We estimate the common true effect, d True effect=d Point estimate= Y i Random error=V i

17. Generic Fixed Effect Y i ~ Normal(d,V i ) where i= 1…….N studies Y i is the observed effect in study i with Variance V i All studies assumed to be measuring the same underlying effect size, d For a Bayesian analysis, a prior distribution must be specified for d

18. Choice of Prior for d Often, amount of information in studies is large enough to render any prior of little importance – therefore choice not critical Often specified as “vague” or “flat” E.g. If meta-analysis is on ln(OR) scale, could specify d~Normal (0, 10 5 ) This states a priori we would be 95% certain that true value of d is between [0±1.96(  10 5 )]

19. Fixed Effect with Prior Y i ~ Normal(d,V i )where i= 1…….N studies d ~ Normal(0, 10 5 ) Models are specified in WinBUGS using formulas similar to this algebra Note: Normal distributions are specified by mean and ‘precision’ –where precision = 1/variance Estimate model parameter using MCMC, rather than inverse weighting of variance

20. Example: Meta-analysis, RCTs of effect of aspirin preventing death after acute MIs StudyAspirin GroupPlacebo Group DeathsTotalDeathsTotal MRC-14961567624 CDP4475864771 MRC-2102832126850 GASP3231738309 PARIS8581052406 AMIS24622672192257 ISIS-21570858717208600 Fleiss. Statistical Methods in Medical Research 1993

21. Example: Calculation: Log(OR) & Variance For MCR-1 OR=(566*67)/ (557*49) = 1.389 Log(ln)OR = 0.3289 Variance lnOR = 1/566 + 1/49 + 1/557 + 1/67 = 0.0389 Note- this is OR for Survival If 2x2 table contains any zeros, common to add 0.5 to those cells before calculations SurviveDie Aspirin56649 Placebo55767

22. Example: Aspirin Data to be Combined StudyORLnOR (Y i )Var(lnOR) (V i )Weight (1/V i ) MRC-11.390.330.0425.71 CDP1.470.390.0424.29 MRC-21.250.220.0248.77 GASP1.250.220.0615.44 PARIS1.250.230.0228.41 AMIS0.88-0.120.01103.92 ISIS-21.120.110.002664.26 Note: ISIS-2 with small variance and large weight (1/0.002)

23. Now It’s Your Turn: Practice using WinBUGS!

24. Launch WinBUGS Click on WinBUGS14.exe Click File-Open Load aspirin FE.odc

25. Components of WinBUGS.odc file Model { } #Data #Starting Values

26. Steps for Running a Model in WinBUGS 1.Make model active. Doodles: If in own window, click title bar. If in compound document, double-click the doodle (should have “hairy” border). Text: Simply highlight the word “model” at the beginning of your model. 2.Bring up Model Specification Tool (menu: Model -> Specification) 3.Click “check model” Should see “model is syntactically correct” in lower left corner of window. 4.Highlight first row of data containing variable labels (if in rectangular format) 5.Click “load data” Should see “data is loaded” in lower left corner of window. 6.If using multiple chains, enter number in “num of chains” box. Otherwise, proceed. 7.Click “compile” Should see “model is compiled” in lower left corner of window. 8.Highlight line containing initial values: list(…) 9.Click “load inits” If using multiple chains, you will need to repeat steps 8-9 for each chain. Should see “model is initialized.” 10.Bring up Sample Monitor Tool (menu: Inference -> Samples) Enter name of each node you wish to monitor and click “set” 11.Bring up Update Tool (menu: Model -> Update) 12.Enter a number of samples to take and click “update.” Should see “model is updating.”

28. Load and Check Model

29. Load and Check Data

30. Compile Model

31. Load Initials

33. Pooled OR: median 1.12 (1.05 to 1.19)

35. Random Effects Model Model –Within studies Y i ~Normal(  i,Vi) –Across studies  I ~Normal(d,  2 ) d=solid line  =dotted lines  2 = variability between studies (heterogeneity) True Mean Effect=d solid line Trial-specific effects=dotted lines Y5Y5 55 Vi

36. Generic Random Effect Y i ~ Normal( ,V i )where i= 1…….N studies  i ~ Normal(d,  2 ) As for fixed effect, Y i is observed effect in study i with variance V i Now study specific effects,  I are allowed to be different from each other and are assumed to be sampled from a Normal distribution with mean d and variance  2 For a Bayesian analysis, a prior distribution is required for  2 as well as for d

37. Choice of Prior for  2 This is a little trickier than for d Variances cannot be negative so Normal distribution is not a good choice Examples in WinBUGS Manual use Uniform distribution. E.g.  ~ Uniform (0,10)  of 10 is massive, because we are working with ORs; even  of 1 or 2 is large Specification of vague priors on variance components is complex and is an active area of research

38. Generic Random Effects Model Load aspirin RE.odc

41. Results of Aspirin RE model Pooled OR: median 1.149 (0.976-1.434) –OR now contains 1 Bayesian CrI wider than classical CI –  2 is random variable and uncertainty is included in pooled result

42. Compare our Two Odds Ratios and CrIs Fixed effects Normal Distribution –OR=1.12 (95% CrI: 1.05, 1.19) Random Effects Normal Distribution –OR=1.15 (95% CrI: 0.97, 1.44)

43. MCMC Basics Now that we’ve run a few models consider sensitivity analyses Sensitivity to prior distributions –esp. important for distributions of variance/precision parameters Sensitivity to initial values –Multiple chains using very different starting values & comparing using Brooks Gelman-Rubin Statistics Length of “burn in”: examine history/trace plots

44. Interpreting Random Effects A single parameter cannot adequately summarize heterogeneous effects Therefore estimation and reporting of  2 is important This tells us how much variability there is between estimates from the population of studies In some instances studies contain both beneficial and harmful effects, so important!

45. Looking to the Future (The Future is Here!) Data Sources RCT1RCT2Obs 1 Routine Care Meta- Analysis General Synthesis Bayes Theorem Combination Evidence Synthesis Clinical Effects Adverse Effects UtilityCosts Model Inputs (w/ uncertainty) Decision Model Utility Cost Tx A Fib Warfarin NoWarfarin No Strk Stroke No Strk Bleed NoBld Bleed NoBld Bleed Utility Cost

46. Questions? bdevine@uw.eduralfonso@uw.edu Attribution for Fleiss example to Keith Abrams, University of Leicester, UK