1. Appraising Patient Management Recommendations: Clinical Practice Guidelines and Decision Analyses
Updated for the third edition of the Users' Guides to the Medical Literature.

2. Objectives
Be able to
Distinguish practice recommendations from other secondary sources of evidence
Evidence literacy
Recognize steps in the development of clinical practice guidelines
Apply Users’ Guides for patient management recommendations
Interpret recommendation grading scales
Evidence numeracy 2

3. Patient Five A’s of EBM Ask Act Acquire Apply Appraise
EBM, evidence-based medicine.
This slide returns to the evidence cycle first explained in the Education Guide “An Approach to Evidence-Based Medicine.” This Education Guide focuses on the “Appraise” step in the evidence cycle, as we appraise patient management recommendations and practice guidelines.
Appraise

4. Clinical Practice Guidelines
What are practice guidelines?
How are guidelines developed?
Assessing recommendations
How should you use guidelines?

5. What Are Practice Guidelines?
Users’ Guides definition
A strategy for changing clinician behavior
Systematically developed statements or recommendations to assist clinician and patient decisions about appropriate health care for specific clinical circumstances

6. What Are Practice Guidelines?
Decision-making aid
Decision guides, reasonable rules, advice, pathway, algorithm, comparisons of alternatives
Statement of authority
Official endorsement, statements of groups
Instrument of control
Costs, privileges, reimbursement

7. What Are Practice Guidelines?
Set boundaries/limits to practice
Clinical standards, appropriateness criteria, minimum acceptable practice, expected performance, practice parameters
Synthesize evidence and expertise
Comprehensive review of evidence, comprehensive analysis, experts assessment, consensus

8. Systematic Reviews vs Guidelines
Unit of analysis = study
Bias control = methods
Synthesis = results
Evidence
Unit of analysis = problem
Bias control = quality of evidence assessment
Synthesis = decision
Evidence and Values

9. Finding Guidelines Citation searching Electronic texts PubMed
National Library of Medicine database
Special markup since 1985
By publication type
Use limits
Publication type = practice guideline
Electronic texts
UpToDate and Dynamed

10. Finding Guidelines Repository searching
Institutional policy collections
Regional guideline collections
National clearinghouses
International registries

11. National Clearinghouses
National Guidelines Clearinghouse
Evidence-based clinical practice guidelines
National Quality Measures Clearinghouse
Evidence-based quality measures and measure sets

12. National Clearinghouses
National Patient Safety Network
Clearinghouse of information about policies and practices affecting patient safety
National Institute for Clinical Excellence
Evidence-based clinical practice guidelines
Clinical practice and public health

13. Outline What are practice guidelines? How are guidelines developed?
Assessing recommendations
How should you use guidelines?

14. Developing Recommendations
Practice guidelines are ideally informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options
To make a recommendation, guideline panelists ideally
Define clinical questions
Select the relevant outcome variables
Retrieve and synthesize all of the relevant evidence
Rate the confidence in the effect estimates
Rely on a systematic approach and consensus to move from evidence to recommendations

15. Developing Recommendations
In general, patient management recommendations are developed in context of clinical practice guidelines
Clinicians may also find guidance originating from a decision analysis

16. Decision Analysis
Systematic approach to decision making under conditions of uncertainty
Involves
Identifying all available alternatives and estimating the probabilities of potential outcomes associated with each alternative
Valuing each outcome
Arriving at a quantitative estimate of the relative merit of each alternative, on the basis of probabilities and values

17. Decision Trees
Most clinical decision analyses are built as decision trees
Decision analysis articles will usually include 1 or more diagrams showing the structure of the decision tree used for the analysis

18. Decision Tree: Example
Abbreviation: LMWH, low-molecular-weight heparin.
This figure shows a simplified decision tree for the scenario of the pregnant woman considering thromboprophylaxis. The patient has 2 options: to use or not use prophylaxis with LMWH. The decision is represented by a square, termed a “decision node.” The lines that emanate from the decision node represent the clinical strategies under consideration. Circles, called “chance nodes,” symbolize the different events that can occur after each clinical strategy. Patients may or may not develop a thrombotic or bleeding event, and the decision analysis requires estimates of the probability of both events. Triangles or rectangles identify outcome states.
Learners can view this decision tree in more detail at

19. Outline What are practice guidelines? How are guidelines developed?
Assessing recommendations
How should you use guidelines?

20. Assessing Recommendations
Is the clinical question clear and comprehensive?
Were the recommendations based on current best evidence?
Are values and preferences associated with the outcomes appropriately specified?
Do the authors indicate the strength of their recommendations?
Is the evidence supporting the recommendations easily understood?
Was the influence of conflict of interests minimized?

21. Is the Question Clear?
Is the recommended intervention clear and actionable?
Is the alternative clear?
Were all patient-important outcomes explicitly considered?

22. Patient-Important Outcomes
Outcomes that patients value directly, in contrast to surrogate outcomes that clinicians may consider important (eg, lipid levels, bone density)
Patient-important outcomes include mortality and morbidity (eg, hospital admission, acute exacerbation of a chronic disease), as well as outcomes such as quality of life and functional status

23. Recommendations and Evidence
Ideally, recommendations are based on current or updated systematic reviews, preferably with meta-analyses
Clinicians should check the date of the literature review and look for a description of the process used to identify and summarize the evidence and judge to what extent the process is credible
Recommendations that do not use best current evidence risk promoting suboptimal or even harmful care

24. Recommendations and Evidence
Systematic review: Identification, selection, appraisal, and summary of primary studies that address a focused clinical question using methods to reduce the likelihood of bias
Meta-analysis: Statistical technique for quantitatively combining the results of multiple studies that measure the same outcome into a single pooled or summary estimate
See also, The Process of a Systematic Review and Meta-analysis, at

25. Considering Values and Preferences
In treatment recommendations, values and preferences should be appropriately specified for each outcome
Clinicians should look for explicit statements regarding the values and preferences used to inform the recommendation

26. Values and Preferences
This term refers to the goals, expectations, predispositions, and beliefs that individuals have for certain decisions and their potential outcomes
Incorporation of patient values and preferences in decision making is central to evidence-based medicine

27. Strength of Recommendations
Trustworthy recommendations specify the strength of the recommendations and also a rating of confidence in effect estimates that support the recommendations (also known as quality of evidence)
Sensitivity analyses are used to explore strength of conclusions that arise from a decision analysis

28. Sensitivity Analysis
Any test of stability of conclusions of a health care evaluation over a range of probability estimates, value judgments, and assumptions about the structure of the decisions to be made
May involve repeated evaluation of a decision model in which one or more of the parameters of interest are varied

29. Grades of Recommendations
There are 3 commonly used approaches to grading recommendations
GRADE (Grading of Recommendations Assessment, Development and Evaluation)
American Heart Association (AHA)
US Preventive Services Task Force (USPSTF)

30. Grades of Recommendations
Directions and strength of recommendations
Abbreviations: AHA, American Heart Association; GRADE, Grading of Recommendations Assessment, Development and Evaluation; USPSTF, US Preventive Services Task Force.
The 3 grading systems included in this figure feature a rating for confidence in effect estimates. Confidence in the effect estimates represents the extent to which the estimates are sufficiently credible to support a particular recommendation, as shown in the figure.
The GRADE approach specifies 4 levels of confidence: high, moderate, low, and very low. The AHA and USPSTF systems specify 3 levels of confidence: A, B, and C in the AHA approach and high, moderate, and low in the USPSTF approach.
The 3 systems share another critical feature: they differentiate between recommendations that should be applied (or avoided) in all, or almost all, patients (ie, strong recommendations) from those that require individualization to the patient’s values, preferences, and circumstances (ie, weak recommendations).

31. Is Supporting Evidence Easily Understood?
For strong recommendations, is the strength appropriate?
For weak recommendations, does the information provided facilitate shared decision making?

32. Strong Recommendations
Message to clinicians is “just do it”
Recommendations that are inappropriately graded as strong may therefore have undesirable consequences

33. Strong Recommendations
High confidence in effect estimates will support strong recommendation if
Desirable consequences considerably outweigh undesirable ones
There is reasonable confidence and limited variability in patients’ values and preferences
Benefits of proposed course of action justify its cost

34. Weak Recommendations
Weak recommendations in particular should explicitly provide key information necessary to act on recommendation
In guidelines, typically found in
Remarks section
Recommendation rationale
Tables that accompany recommendation
GRADE Working Group and Cochrane Collaboration designed summary-of-findings table specifically for this purpose

35. Summary-of-Findings Table
In a practice guideline developed according to the GRADE method, the summary-of-findings table provides confidence ratings for all important outcomes and associated estimates of relative and absolute effects
Summary-of-findings tables can facilitate shared decision making
Learners can view a summary-of-findings table relevant to an example used in Chapter 26 of the Users’ Guides to the Medical Literature at

36. Appraising Conflicts of Interest
Judgments involved in interpretation of evidence and the decision on the final recommendation may be vulnerable to conflicts of interest

37. Conflicts of Interest
Exist when investigators, authors, institutions, reviewers, or editors have financial or nonfinancial relationships with other persons or organizations (such as study sponsors) that may inappropriately influence their interpretation or actions
Conflicts of interest can lead to biased design, conduct, analysis, and interpretation of study results and to bias in review articles and opinion pieces

38. Appraising Conflicts of Interest
Clinicians can check the conflict of interest statements of guideline panelists or decisions analysts, usually found at the beginning or end of a publication or in a supplementary file
Clinicians should also check what strategies were implemented to manage these conflicts of interest

39. Outline What are practice guidelines? How are guidelines developed?
Assessing recommendations
How should you use guidelines?

40. Strong Recommendations
If panel’s assessment is astute, clinicians can apply strong recommendations to all or almost all patients in all or almost all circumstances without a review of the underlying evidence and without a detailed discussion with the patient
Also true for decision analysis when utility of one alternative is substantially greater than the other and this relative utility is robust to sensitivity analyses
It is important to note that there will always be idiosyncratic circumstances in which clinicians should not adhere to even strong recommendations. For examples of such circumstances, please refer to Chapter 26 of the Users’ Guides to the Medical Literature,

41. Weak Recommendations
Typically sensitive to a patient’s values and preferences
A shared decision-making approach that involves a discussion addressing potential benefits and harms is the optimal way to ensure decisions reflect best evidence and the patient’s values and preferences
A solid understanding of the evidence is necessary for clinicians when using weak recommendations

42. Original slides created by Robert Hayward, MD, Centre for Health Evidence
Updated by Gordon Guyatt, MD, Kate Pezalla, MA, and Annette Flanagin, RN, MA

43. Terms of Use: Users Guides to the Medical Literature Education Guides
PowerPoint Usage Guidelines
JAMAevidence users may display, download, or print out PowerPoint slides and images associated with the site for personal and educational use only. Educational use refers to classroom teaching, lectures, presentations, rounds, and other instructional activities, such as displaying, linking to, downloading, printing, and making and distributing multiple copies of said isolated materials in both print and electronic format. Users will only display, distribute, or otherwise make such PowerPoint slides and images from the applicable JAMAevidence materials available to students or other persons attending in-person presentations, lectures, rounds, or other similar instructional activities presented or given by User.
Commercial use of the PowerPoint slides and images are not permitted under this agreement. Users may modify the content of downloaded PowerPoint slides only for educational (non-commercial) use; however, the source and attribution may not be modified. Users may not otherwise copy, print, transmit, rent, lend, sell, or modify any images from JAMAevidence or modify or remove any proprietary notices contained therein, or create derivative works based on materials therefrom. They also may not disseminate any portion of the applicable JAMAevidence site subscribed to hereunder through electronic means except as outlined above, including mail lists or electronic bulletin boards.