1. Plan GRADE backgroundGRADE background confidence in estimates (quality of evidence)confidence in estimates (quality of evidence) evidence profilesevidence profiles confidence in estimates and recommendationsconfidence in estimates and recommendations

2. Dilemma: proliferation of systems Solution: common international grading system? GRADE (Grades of recommendation, assessment, development and evaluation)GRADE (Grades of recommendation, assessment, development and evaluation) international groupinternational group –Australian NMRC, SIGN, USPSTF, WHO, NICE, Oxford CEBM, CDC, CC ~ 30 meetings over last twelve years~ 30 meetings over last twelve years (~10 – 50 attendants)(~10 – 50 attendants)

3. 60+ Organizations 3 2005 2006 2007 2008 2009 2010 2011

4. GRADE uptake

5. Confidence in estimate (quality of evidence) no confidence totally confident High Moderate Low Very Low Randomized trials start high confidence Observational studies start low confidence

6. Determinants of confidence risk of bias –concealment – blinding – loss to follow-up imprecision –wide confidence intervals publication bias

7. Relative Risk with 95% CI for Vitamin D Non-vertebral Fractures

8. 25% 50% 75% No worries Some concern Serious concern Why are we Pooling?

9. Relative Risk with 95% CI for Vitamin D Non-vertebral Fractures

10. Quality judgments: Directness populationspopulations –older, sicker or more co-morbidity interventionsinterventions –warfarin in trials and community outcomesoutcomes –important versus surrogate outcomes –glucose control versus CV events

11. What can raise confidence? large magnitude can upgrade one levellarge magnitude can upgrade one level –very large two levels common criteriacommon criteria –everyone used to do badly –almost everyone does well –quick action hip replacement for hip osteoarthritiship replacement for hip osteoarthritis mechanical ventilation in respiratory failuremechanical ventilation in respiratory failure

12. Quality assessment criteria

13. Nonfatal MI – Fixed Effects

14. Quality Assessment Summary of Findings Quality Relative Effect (95% CI) Absolute risk difference Outcome Number of participants (studies) Risk of Bias ConsistencyDirectnessPrecision Publication Bias Myocardial infarction 10,125 (9) No serious limitations No serious imitations No serious limitations Not detected High 0.71 (0.57 to 0.86) 1.5% fewer (0.7% fewer to 2.1% fewer) ) Beta blockers in non-cardiac surgery

15. Mortality – Fixed Effects

16. Quality Assessment Summary of Findings Quality Relative Effect (95% CI) Absolute risk difference Outcome Number of participants (studies) Risk of Bias ConsistencyDirectnessPrecision Publication Bias Myocardial infarction 10,125 (9) No serious limitations No serious imitations No serious limitations Not detected High 0.71 (0.57 to 0.86) 1.5% fewer (0.7% fewer to 2.1% fewer) Mortality 10,205 (7) No serious limitations Possiblly inconsistent No serious limitations Imprecise Not detected Moderate or low 1.23 (0.98 – 1.55) 0.5% more (0.1% fewer to 1.3% more) Beta blockers in non-cardiac surgery

17. Stroke – Fixed Effects Total events 75/10,290

18. Quality Assessment Summary of Findings Quality Relative Effect (95% CI) Absolute risk difference Outcome Number of participants (studies) Risk of Bias ConsistencyDirectnessPrecision Publication Bias Myocardial infarction 10,125 (9) No serious limitations No serious imitations No serious limitations Not detected High 0.71 (0.57 to 0.86) 1.5% fewer (0.7% fewer to 2.1% fewer) Mortality 10,205 (7) No serious limitations Possiblly inconsistent No serious limitations Imprecise Not detected Moderate or low 1.23 (0.98 – 1.55) 0.5% more (0.1% fewer to 1.3% more) Stroke 10,889 (5) No serious limitaions No serious limitations Not detected High 2.21 (1.37 – 3.55) 0.5% more (0.2% more to 1.3% more0 Beta blockers in non-cardiac surgery

19. Significance of strong vs weak variability in patient preferencevariability in patient preference –strong, almost all same choice (> 90%) –weak, choice varies appreciably interaction with patientinteraction with patient –strong, just inform patient –weak, ensure choice reflects values use of decision aiduse of decision aid –strong, don’t bother –weak, use the aid quality of care criterionquality of care criterion –strong, consider –weak, don’t consider

20. Strength of Recommendation strong recommendation –benefits clearly outweigh risks/hassle/cost –risk/hassle/cost clearly outweighs benefit what can downgrade strength? low confidence in estimates close balance between up and downsides

21. Conclusion clinicians, policy makers need summariesclinicians, policy makers need summaries –quality of evidence explicit rulesexplicit rules –transparent, informative GRADEGRADE –simple, transparent, systematic