1. Searching for Evidence

2. Task 1 Find a full text document from a reference The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group. Major outcomes in moderately hypercholesterolemic, hypertensive patients randomized to pravastatin vs usual care: the antihypertensive and lipid- lowering treatment to prevent heart attack trial (ALLHAT-LLT). JAMA. 2002;288:2998-3007.

3. Task 2 Find the best evidence concerning the use of aspirin for the primary prevention of cardiovascular disease in people with diabetes

4. ATHENS ACCOUNT To get a NHS Athens account, username and password. Register for free at: https://register.athensams.net/nhs/nhseng/ https://register.athensams.net/nhs/nhseng/

5. Task 3 NNT - Revision From the diagram on the next slide Calculate: Relative Risk Absolute Risk Reduction Numbers Needed to Treat

7. Task 4 NNT Calculation In the Statins Metaanalysis over a mean 4.3 years: Baseline risk of a major coronary event = 5.7% Relative Risk Reduction is 29.2% Calculate Absolute Risk Reduction and Numbers Needed to Treat (for 4.3 years)

8. Task 5 From the NNT found in the previous task Calculate the drug cost to prevent one coronary event using Simvastatin (£1.37 for 28 tablets)

9. Task 6 The JUPITER trial of Rosuvastatin 20mg vs placebo (in 17802 healthy men & women LDL<3.4 mmol/l, high sensitivtiy CRP 2.0+ mg/l) reported: 0.77 vs 1.36 events per 100 person-years HR 0.56 (0.46-0.69) p<0.00001 Rosuvastatin 20mg costs £26.02 for 28 tabs What was the drug cost to prevent 1 event in this trial?

10. Systematic Reviews & Guidelines Chris Lewis Apr 2009

11. Types of “paper” research evidence Primary studies – Case studies – Experiments – Surveys – Clinical Trials Secondary studies – Non-systematic reviews – Systematic reviews Meta-analyses Guidelines Decision analyses Economic analyses

12. AdvantagesDisadvantages Evidence-based Guideline Summarises all relevant research about all possible interventions for a clinical problem. Explores benefits and harms. May become out-of-date quickly. Expert opinion often fills gaps in evidence. Systematic ReviewSummarises all research about an intervention. Usually only one of several possible interventions is considered. May not explore benefits vs harms. Primary StudyVery specific informationNot comprehensive Types of evidence

13. Systematic Review A systematic review is a literature review focused on a single question that tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question.

14. Meta-analysis A systematic review in which the data from the primary studies is sufficiently similar that they can be analyzed as if they were the same trial. They provide higher statistical power to detect an effect and a more powerful estimate of true effect size than individual studies.

15. How to conduct a systematic review Synthesize, analyze & present results Results / Conclusions Assess studies At least 2 appraisersMethods Find all relevant studies Search +++Methods Formulate Question PICOTitle / Abstract

16. FAST critical appraisal 4 FAST Questions for appraising systematic reviews: Finding: Did they find all relevant studies? Appraisal: Did they select good studies? Synthesis: When the studies are put together what do they mean? Transferability: Are the conclusions applicable to my patient?

17. Critical Appraisal Q1 – Formulate Question Did the study address a focused clinical question? Population Intervention Comparator Outcome Should be found in the title, abstract or introduction

18. Critical Appraisal Q2 – Find all relevant studies Were the inclusion/exclusion criteria appropriate? Clearly defined before the literature search Should specify PICO May specify type of study (eg RCT) Should be found in the methods section

19. Critical Appraisal Q3 – Find all relevant studies Is it unlikely that important relevant studies were missed? Search strategy should include: Major databases: Medline, EMBASE, Cochrane Reference lists from relevant studies Inquiry into unpublished studies Foreign language publications MESH terms and text words Methods section should describe search strategy Results section should state number of studies retrieved and number of studies excluded with reasons

20. Copyright restrictions may apply. Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313. Literature review flowchart

21. Publication Bias ‘Negative’ studies less likely to be published than ‘ positive’ studies In a follow-up of 737 studies submitted to the ethics committee at the Johns Hopkins hospital positive studies were 2.5 times more likely to be SUBMITTED than negative studies (Dickersin, JAMA, 1992) All trials registered at inception Unethical not to make results available

22. Critical Appraisal Q4 – Assess studies Were the included studies valid? Should describe how quality of included studies was assessed Criteria for quality assessment should be pre- determined Criteria for quality assessment should be appropriate to the question asked Methods section should describe assessment process and quality criteria Results section should give information on quality of the individual studies

23. Critical Appraisal Q5 – Assess studies Were assessments of studies reproducible? Assessed independently by 2+ reviewers Level of agreement between reviewers Procedure for dealing with disagreement Methods should describe how and by who assessments were done Results should show level of agreements

24. Critical Appraisal Q6 – Analysis/Results Were the results similar from study to study? Ideally results should be similar (homogeneous) Significance of heterogeneity may be assessed Reasons for heterogeneity should be explored Results section. Forest plot should show chi 2 test for heterogeneity

25. Meta-analysis (Forest) plot 1.How many studies are there? 2.How many studies favour treatment? 3.How many studies are statistically significant? 4.Which is the largest study? 5.Which is the smallest study? 6.What is the combined result?

26. Assessing heterogeneity “Eyeball” test: In the Forest plot a vertical line running through the combined OR should cross the horizontal lines (95% CI) of all the individual studies

27. Assessing heterogeneity Chi 2 test or Cochran Q: If chi 2 is statistically significant (p<0.1) there is definite heterogeneity If chi 2 not statistically significant (p>0.1) and Q/df >1 there is possible heterogeneity If chi 2 not statistically significant (p>0.1) and Q/df >1 heterogeneity is very unlikely

28. Explaining heterogeneity Variation in population studied Variation in intervention Variation in outcome measures Variation in study methods

29. Copyright restrictions may apply. Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313. Plotted relative risk ratios (RRs) (95% confidence intervals [CIs]) for major coronary events

30. Copyright restrictions may apply. Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313. Plotted relative risk ratios (RRs) (95% confidence intervals [CIs]) for major cerebrovascular events

31. Critical appraisal – Question 7 Are conclusions supported by data? Biased interpretation or emphasis of results is surprisingly common Best to form your own conclusions from the data before reading authors conclusions Then consider the reasons for any differences between your conclusions and the authors’

32. Critical Appraisal Q8 Are the results clinically relevant? Assuming we have a statistically significant benefit from treatment: Is the size of the benefit worthwhile? Are the results applicable to my patient?

33. Funding Bias? Funding / employment of authors Drug company sponsorship

34. Results Statins reduce risks by: Coronary events 29.2% (16.7-39.8%) p<.001 Cerebrovascular events 14.4% (2.8-24.6%) p0.02 Revascularisations 33.8% (19.6-45.5%) p<.001 Cardiovascular death 22.6% (0.56-1.08) p.13 All deaths 0.92 (0.84-1.01) p.09

35. Inclusion Criteria / Baseline characteristics TrialAgeMean age Sex % male GP / HospChol mmol/lOther WOSCOPS45 - 6455100GP (W.Scotland) F LDL >4.5 <6.0 CVD 16% Smoke 44% AFCAPS/T exCAPS M45-73 F55-73 5885Texas ?Hosp TC 4.65-6.82 Trig<=4.52 102800 screened 6600 randomised PROSPER70-827542?Hosp (Scot/Ire/Net hrlnds) TC 4.0-9.0Smo/HT/Diab ALLHAT- LLT 55+6651GP (N.America) LDL >=3.1 <=4.9 Trig <3.95 Hypertension +1 other RF ASCOT- LLA 40 - 796381GP (UK/Scand) TC <=6.5Hypertension +3 other RF’s HPS40 - 80NA Hosp (UK)F TC >=3.5Diabetes CARDS40 - 756168GP & Hosp (UK/Ireland) LDL <=4.14 F Trig <=6.78 Diabetes CARDS required at least 1 of retinopathy, albuminuria, smoking, hypertension

36. Influence of Diabetes TrialRR Major Coronary Events% Diabetic WOSCOPS 19950.70 (0.58-0.85)1.0 AFCAPS/TexCAPS 19980.60 (0.43-0.83)3.8 PROSPER 20020.91 (0.71-1.15)12.2 ALLHAT-LLT 20020.91 (0.79-1.04)34.4 ASCOT-LLA 20030.65 (0.50-0.83)24.3 HPS 20030.57 (0.41-0.79)100 CARDS 20040.53 (0.35-0.82)100

37. Systematic Reviews Advantages: Larger numbers with greater statistical power Robustness across differing trial populations, drugs within a class Disadvantages: Small, but consistent, biases may produce an invalid conclusion of real effect

38. Resources Oxford Centre for Evidence-based Medicine http://www.cebm.net/ http://www.cebm.net/ The Cochrane Collaboration http://www.cochrane.org/ http://www.cochrane.org/

39. Formula for caluclating NNT from OR if Patient Expected Event Rate is known

40. GUIDELINES The AGREE Instrument Appraisal of Guidelines for REsearch & Evaluation The AGREE Collaboration – June 2001 www.agreecollaboration.org

41. The AGREE Instrument 23 items organised in six domains Each item is rated on a 4-point score scale 1.Strongly disagree 2.Disagree 3.Agree 4.Strongly agree Overall assessment Strongly recommend Recommend (with provisos/alterations) Would not recommend Unsure

42. Domain 1: Scope & Purpose 1.Overall objective(s) 2.Clinical question(s) 3.Target population should all be specifically described

43. Domain 2: Stakeholder Involvement 4.Guideline development group includes individuals from all relevant professional groups 5.Patients’ views & preferences sought 6.Target users clearly defined 7.Piloted by end users

44. Domain 3: Rigour of development 8.Systematic search for evidence 9.Criteria for selecting evidence described 10.Methods for formulating recommendations described 11.Health benefits, unwanted effects and risks considered in formulating recommendations 12.Explicit link between recommendations and evidence 13.External review by experts before publication 14.Procedure for update provided

45. Domain 4: Clarity & Presentation 15.Recommendations specific & unambiguous 16.Management options clearly presented 17.Key recommendations easily identifiable 18.Guideline supported by tools for application

46. Domain 5: Applicability 19.Potential organisational barriers in applying the recommendations discussed 20.Cost implications considered 21.Key review criteria for monitoring/audit presented

47. Domain 6: Editorial Independence 22.Editorially independent from funding body 23.Conflicts of interest of guideline development members recorded