Clinical Practice Guidelines: Can we fix Babel? Eddy Lang Department Chair, Emergency Alberta Health Services Associate Professor University of Calgary

Enigma What has 2 Brazilian flags and makes the most delicious cocktail in the world?

Disclosures GRADE bias Member of GRADE working group –Projects –Publications –Consultancies

PCBE Model Critical Appraisal Guideline Implementation Interpreting study results

Objectives 1.What is GRADE? 2.Why did it come about? 3.How does GRADE address limitations in CPGs? 4.How does knowledge of GRADE improve patient care

The Promise of Guidelines Evidence Synthesis Reduced variation in care Unbiased Guidance Improved patient outcomes Improved system efficiency Benchmarks

The Disappointment of Guidelines Biased / conflicts of interest Study design drives strength of evidence despite limitations Anchored only to evidence Overwhelming Confusing Directionless Not implemented

What goes into a CPG? Panelists Values and preferences Evidence Synthesis

How do CPGs inform and provide guidance? Quality of evidence Graded recommendations

IOM Report Be based on a systematic review of the existing evidence; Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups; Consider important patient subgroups and patient preferences, as appropriate

Evidence-based clinical decisions Research evidence Patient values and preferences Clinical state and circumstances Expertise Haynes et al. 2002

IOM Report part 2 Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest; Provide a clear explanation of the logical relationships between alternative care options and health outcomes Provide ratings of both the quality of evidence and the strength of recommendations; and Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.

AGREE Scores

Recommendations all over the map

GRADE uptake

24 Before GRADE Level of evidence I II III IV V Source of evidence SR, RCTs Cohort studies Case-control studies Case series Expert opinion A Grades of recomend. B C D

I B IIVIII GRADE is outcome-centric Quality: High Quality: Moderate Quality: Low Older systems Outcome #1 Outcome #2 Outcome #3 GRADE

Where GRADE fits in Prioritize problems, establish panel Find/appraise or prepare: Systematic review Searches, selection of studies, data collection and analysis (Re-) Assess the relative importance of outcomes Prepare evidence profile: Quality of evidence for each outcome and summary of findings Guidelines: Assess overall quality of evidence Decide direction and strength of recommendation Draft guideline Consult with stakeholders and / or external peer reviewer Disseminate guideline Implement the guideline and evaluate GRADE

28 GRADE: Quality of evidence GRADE defines 4 categories of quality: –High –Moderate –Low –Very low The extent to which our confidence in an estimate of the treatment effect is adequate to support a particular recommendation.

Determinants of quality What lowers quality of evidence? 5 factors: Methodological limitations Inconsistency of results Indirectness of evidence Imprecision of results Publication bias RCTs start high Observational studies start low

Cochrane Risk of bias graph in RevMan 5 30

Determinants of quality What raises quality of evidence? 3 factors: Addressing confounding Dose- response Large effect size RCTs start high Observational studies start low

GRADE Recommendations Strong recommendations –strong methods –large precise effect –few downsides of therapy –expect non-variant clinician and patient behavior diminished role for clinical expertise –focus on implementation & barriers focused role of patient values and preferences –emphasis on compliance and barriers Weak recommendations –weak methods –imprecise estimate –small effect –substantial downsides –expect variability in clinician and patient actions clinical expertise important –focus on decision-making and implementation patient values and preferences important –focus on determining values and preferences relative to decision Patient Values and Preferences Research Evidence Clinical Experience

Moving from Evidence to Recommendations

Factors influencing strength of recommendations Strength of evidence Balance between benefits and harms Uncertainty about values and preferences Resource use

Strength of recommendation “The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.” Strong or weak/conditional

Quality of evidence & strength of recommendation GRADE separates quality of evidence from strength of recommendation Linked but no automatism Other factors beyond the quality of evidence influence our confidence that adherence to a recommendation causes more benefit than harm

GRADE evidence profile

Intermediate Risk Pulmonary Embolism Thrombolytic therapy with tPA?

GRADE Quiz #1 For the outcome of in-hospital mortality for intermediate risk patients treated with thrombolytic agents 16 RCTs little to no risk of bias 2000 patients but only 65 deaths I-squared – 15% - minimal heterogeneity No publication bias Many different lytic agents used

GRADE Quiz #2 Helmets in motorcycle safety and mortality in motor vehicle accidents 5 observational studies with 3000 patients Injury severity controlled OR 3.2 with narrow confidence intervals favoring survival with helmet No other risk of bias identified

Guideline Exercise