1. Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

2. Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date approx 75 new trials published every day Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

3. Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date MEDLINE 2010 2,000 articles / day approx 75 new trials published every day Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

4. About 1/3 of worthwhile evidence is eventually refuted or attenuated About 10% of published evidence is worth reading About 1/2 of relevant evidence is not implemented

6. 6 Rapid critical appraisal using GATE Rod Jackson University of Auckland, NZ August 2011

7. Graphic Approach To Evidence Based Practice Graphic Approach To Epidemiology Graphic Appraisal Tool for Epidemiological studies

8. 8 the GATE frame the shape of every epidemiological study 8

9. 9 British doctors non-smokerssmokers Lung cancer yes no 5 years smoking status measured Longitudinal (cohort) study

10. 10 British doctors non-smokerssmokers Lung function normal abnormal smoking status measured Cross-sectional study

11. 11 British doctors placeboaspirin Myocardial infarction yes no 5 years Randomised to aspirin or placebo RCT

12. 12 Middle-aged US women Breast cancer Mammogram negative yes no Test applied Clinical use of a diagnostic test Mammogram positive

13. 13 Middle-aged US women no Breast cancerBreast cancer mammogram positive negative Diagnostic test accuracy study

14. 14 GATE: Graphic Appraisal Tool for Epidemiological studies 1 picture, 2 formulas & 3 acronyms 14

15. 15 One picture: the GATE frame every epidemiological study hangs on the GATE frame

16. 16 Participants Exposure GroupComparison Group Outcomes Time P E C O T The 1 st acronym = PECOT : the 5 parts of every epidemiological study

17. 17 Lewis RT et al. Should antibiotic prophylaxis be used routinely in clean surgical procedures: A tentative yes. Surgery 1995;118:742-7.

18. 18 Background. The incidence of surgical site infection (SSI) after clean surgical procedure is regarded as too low for routine antibiotic prophylaxis. But risk of SSI can be as high as 20%. We assessed the value of prophylactic cefotaxime in patients stratified for risk of SSI in a double-blind RCT. Methods. Patients having clean elective operations were stratified for risk & randomized to receive IV cefotaxime 2 gm or placebo before operation & followed for 4-6 weeks for SSI. Results. The 378 of 775 patients who received cefotaxime had 70% fewer SSIs than those who did not --Mantel-Haenszel risk ratio (MH-RR) 0.31; 95 % CI 0.11 to 0.83. Benefit was clear in the 616 low risk patients--0.97% versus 3.9% SSI (MH-RR 0.25, CI 0.07 to 0.87, p = 0.018), but only a trend was seen in 136 high risk patients--2.8% versus 6.1% SSI (MH-RR 0.48, CI 0.09 to 2.5). Conclusions. The results indicate clear benefit for routine antibiotic prophylaxis in clean surgical procedures. High risk patients need more study. 18

19. 19

20. 20 P E C O T 1 st critical appraisal task: describe studys design by hanging on GATE frame using PECOT acronym

21. 21 Participants Study Setting Eligible Participants Participants P

22. 22 Participants Study Setting: patients admitted to QE Hospital, Montreal, Canada (1992-5?) Eligible Participants: undergoing clean surgery or simple cholecystectomy Participants: 633 (?consecutive eligible patients) P Lewis Trial

23. 23 Exposure & Comparison Groups Exposure or Intervention Group (EG) Comparison or Control Group (CG) EG CG

24. 24 Exposure & Comparison Groups: low risk group Exposure or Intervention Group (EG): 2g cefotaxime IV pre- op Comparison or Control Group (CG): Identical placebo IV 316 317 633

25. 25 Exposure & Comparison Groups: low risk group Exposure or Intervention Group (EG): 2mg cefotaxime IV pre-op Comparison or Control Group (CG): Identical placebo IV 316 317 633 308* * With complete follow-up

26. 26 Outcomes (O) O ab cd yes no Dis-ease

27. 27 Outcomes (O) Primary Outcome (O) O a= 3b= 12 cd yes no Surgical site infection (SSI) 316 317 633

28. 28 Time (T) T incidence prevalence

29. 29 Time (T) T= time from initiation of treatment to end of follow-up incidence Outcome: SSI 316 317

30. 30 Study design: GATE frame & PECOT Participants Exposure GroupComparison Group Outcomes Time P E C O T

31. 31 Participants Exposure Group: IV cefotaxime Comparison Group: IV placebo Outcomes: SSI Time: Up to 6 wks post-op 633 Lewis 316 317 Setting: QE Hospital, Montreal Eligible: clean surgery or cholecystectomy 3 12 308

32. 32 Questions?

33. 33 The 1 st formula: study analyses Occurrence (risk) of disease = Numerator ÷ Denominator D N

34. 34 Denominator (Participants) D N Numerator (Outcomes) Occ = N÷D All epidemiological studies involve measuring the OCCURRENCE of outcomes

35. 35 Denominator (Participants) D N Numerator (Outcomes) Occ = N÷D (T?) All epidemiological studies involve measuring the OCCURRENCE of outcomes T During what period of time (T) was N measured? (incidence)

36. 36 Denominator (Participants) D N Numerator (Outcomes) Occ = N÷D (T?) All epidemiological studies involve measuring the OCCURRENCE of outcomes T At what point in time (T) was N measured? (prevalence)

37. 37 The 1 st formula: Occurrence (risk) = Numerator ÷ Denominator O DEDE DCDC NENE NCNC P T T Exposed Group Comparison Group

38. 38 P EG CG O Exposure Group (EG) Numerator 1: a Comparison Group (CG) ab cd Numerator 2: b 2 nd appraisal task: describe analyses by hanging numbers on the GATE frame and calculating occurrences in exposure & comparison groups Denominator 1:Denominator 2:

39. 39 Occurrence = N ÷ D P EG CG O Denominator 1: Exposure Group EG Numerator 1: a Denominator 2: Comparison Group CG ab cd Numerator 2: b Exposure Group Occurrence: EGO = a ÷ EG Comparison Group Occurrence: CGO = b ÷ CG

40. 40 Occurrence = N ÷ D P EG CG O Denominator 1: Exposure Group EG Numerator 1: a Denominator 2: Comparison Group CG ab cd Numerator 2: b Exposure Group Occurrence: EGO = a ÷ EG Comparison Group Occurrence: CGO = b ÷ CG

41. 41 Occurrence = N ÷ D P EG CG O Denominator 1: Exposure Group EG Numerator 1: a Denominator 2: Comparison Group CG ab cd Numerator 2: b Exposure Group Occurrence: EGO = a ÷ EG Comparison Group Occurrence: CGO = b ÷ CG

42. 42 Calculate EGO & CGO for SSI in low risk group P EG CG O ab cd Denominator 1: Exposure Group EG = 316 Numerator 1: a = 3 Denominator 2: Comparison Group CG = 317 Numerator 2: b = 12 EGO = 3/316= 9.5/1000 at 6 weeks CGO = 12/317 = 37.9/1000 at 6 weeks ITT (intention to treat) analysis

43. 43 Calculate EGO & CGO for SSI in low risk group P EG CG O ab cd Denominator 1: Exposure Group EG = 308 Numerator 1: a = 3 Denominator 2: Comparison Group CG = 308 Numerator 2: b = 12 EGO = 3/308= 9.7/1000 at 6 weeks CGO = 12/308 =39/1000 at 6 weeks OT (on treatment) or per-protocol analysis

44. 44 Describing differences between occurrences Relative difference or Relative Risk = EGO ÷ CGO Absolute Difference or Risk Difference = EGO - CGO Number Needed To Treat (NNT) = 1 ÷ RD

45. 45 Describing differences between occurrences (SSI in low risk patients) Relative difference or Relative Risk = EGO ÷ CGO Absolute Difference or Risk Difference = EGO - CGO Number Needed To Treat (NNT) = 1 ÷ RD = 9.5/1000 ÷ 37.9/1000 = 0.25 = 9.5/1000 - 37.9/1000 = -28.4/1000 = 1 ÷ (- 28.4 /1000) = - 1000/28.4 = 35 if 35 patients were given IV cefotaxime pre-op, there would be 1 fewer SSI up to 6 weeks post-op

46. 46 Study analyses its all about EGO & CGO

47. 47 Questions?

48. 48 P EC O T PECOTPECOT * Paul Glasziou The 2 nd acronym = RAMBO* : assessing bias strength of study Recruitment Allocation Maintenance Blind or Objective outcomes measurement 48

49. 49 P EC O T PECOTPECOT * Paul Glasziou The 2 nd acronym = RAMBO* : assessing non random error (i.e. bias) Recruitment Allocation Maintenance Blind or Objective outcomes measurement 49

50. 50 P EC O T PECOTPECOT 3 rd appraisal task: assess the degree of bias by applying the RAMBO acronym Recruitment Allocation Maintenance Blind or Objective outcomes measurement

51. 51 R AMBO E C O T were Recruitment processes appropriate to study goals? P Study setting & eligibility criteria well described? e.g. Recruit random/representative sample OR consecutive eligibles OR volunteers from advertisements Participants representative of eligibles? Prognostic/risk profile appropriate to study question? P Study setting Eligible people

52. 52 EG CG O T RCT: Allocate randomly by investigators (e.g drugs) EG CG O T Cohort: Allocate by measurement (e.g. smoking) R A MBO: A is for Allocation were EG & CG similar at baseline? Was Allocation to EG & CG successful?

53. 53 RA M BO EG CG O T were Participants Maintained as allocated? did most participants remain in allocated groups (EG & CG) P Participants &/or investigators blind to exposure (and comparison exposure)? Compliance high & similar in EG & CG? Contamination low & similar in EG & CG? Co-interventions low & similar in EG & CG? Completeness of follow-up high & similar in EG & CG?

54. 54 RAM BO EG CG O T Were outcomes measured Blind or Objectively? P If outcome measurements not Objective (eg. automated or definitive) were investigators Blind to exposure (and comparison exposure)

55. 55 The 4 (GATE) study biases P E C O T Recruitment bias Allocation bias Maintenance bias Outcomes Measurement bias

56. 56 Questions?

57. 57 The 2 nd formula: assessing random error Random error = 95% Confidence Interval(1.96 x Standard Error) 57

58. 58 4 th appraisal task: assess degree of random error in study findings using the 2 nd formula Random error = 95% Confidence Interval For the Outcome SSI (low risk group) : EGO = 9.5/1000; (95% CI = 3.2 to 27.5) CGO = 37.9/1000; (95% CI = 21.8 to 65) EGO÷CGO = 0.25 (0.07 to 0.88) EGO-CGO = -28.4 (-52 to -4.8) NNT = -35 (-19 to -211) 58

59. 59 Excel CATs & paper Gate-lites There is a GATE for every study design www.epiq.co.nz 59

60. 60 Final appraisal task: search for & appraise SRs / meta-analyses using 3 rd acronym (FAITH) F ind appropriate studies? A ppraise selected studies? I nclude only valid studies? T otal-up (synthesise) appropriately? H eterogeneity adequately addressed? 60

61. 61 Systematic Reviews There are 4 Cochrane SRs on this topic and the findings are not consistent

62. Using GATE as a framework for evidence based practice

63. The first 4 steps of EBP 1. Ask a focused question. 2. Access (systematically search for) epidemiological evidence to help answer question. 3. Appraise evidence found for its validity, effect size, precision (ideally all the relevant evidence) 4. Apply the evidence: a. amalgamate the valid evidence with other relevant information (patient/community values, clinical/health issues, & policy context) and make an evidence-based decision; and b. act (implement) the decision in practice

64. EBP Step 1: Ask- turn your question into a 5-part PECOT question Participants (the patient problem) Exposure (e.g. a therapy) Comparison (there is always an alternative! - another therapy or no treatment… Outcome (e.g. a disease you want to prevent or manage) Time frame (over which you expect a result)

65. 65 EBP Step 2: Access the evidence – use PECOT to choose search terms Participants (the patient problem) Exposure (e.g. a therapy) Comparison (there is always an alternative! - another therapy or no treatment… Outcome (e.g. a disease you want to prevent or manage) Time frame (over which you expect a result) 65

66. EBP Step 3: Appraise the evidence using the best evidence from epidemiology to help inform decisions more critically (using GATE) more systematically (using FAITH)

67. EBP Step 4: APPLY the evidence by: a. AMALGAMATING the relevant information & making an evidence-based decision: the X-factor ©

69. Evidence Clinical / health considerations Policy issues Patient / community preferences X-factor: making evidence-based decisions X pertise: putting it all together the art of practice