Critical Appraisal of Clinical Practice Guidelines Manit Srisurapanont, MD Department of Psychiatry Chiang Mai University For Evidence-Based Medicine Workshop, Chiang Mai, Nov 17-18, 2011

Outline Practice Guidelines Assessing Recommendations (Gayatte et al. Users’ Guide to the Medical Literature: a Manual for Evidence-Based Medicine, 2008) Appraisal of Guidelines for Research & Evaluation II (AGREE II, 2009)

Practice Guidelines Are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances  Play an important role in health policy formation Practice guidelines are evidence-based if they undertake a review of the literature and link their concluding recommendations to the evidentiary base identified through the literature search 

From evidence to practice + clinical expertise** *Haynes RB, Haines A. BMJ 1998;317:273-6. **Haynes RB, et al. EBM 2002;7:36-8.

Assessing Recommendations: 1 1. Do the recommendations consider all relevant patient groups, management options, and possible outcomes? All relevant patient groups, eg, low risk/high risk, more/less susceptible to adverse effects All relevant management options, eg, surgical/medical , no-treatment option All patient-important outcomes, eg, morbidity and mortality, quality of life, toxicity/adverse effects, cost to the patient or to society Guyatt G, et al. User’s Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2008

Assessing Recommendations: 2 2. Are there systematic reviews of evidence that estimate the relative effect of management options on relevant outcomes? For example, hormone replacement therapy: the incidence of hip fracture, breast cancer, endometrial caner, myocardial infarction, stroke, and dementia , as well as quality of life The guideline developers’ systematic review must summarize the quality of the evidence on which they base their recommendations

Assessing Recommendations: 3 3. Is there an appropriate specification of values and preferences associated with outcomes? It may be that the greater participation by methodologists, frontline clinicians, and members of the general public would lead to guidelines more in keeping with the public interest It is important that authors of guidelines report the principal sources of judgments and the method of seeking consensus on values and preferences

Assessing Recommendations: 4 4. Do the authors indicate the strength of their recommendations? The grades of recommendation, assessment, development, and evaluation (GRADE) system: Strong vs. weak recommendation High-, moderate-, low- and very-low-quality evidence For example: Strong recommendation, high-quality evidence: Benefits clearly outweigh risk and burden, or vice versa, RCTs without important limitations or overwhelming evidence from observational studies

AGREE II The Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument was developed to address the issue of variability in guideline quality The AGREE instrument is a tool that assesses the methodological rigor and transparency in which a guideline is developed The original AGREE instrument (2001) has been refined, which has resulted in the new AGREE II (2010) and includes a new User’s Manual AGREE II comprises 23 items in 6 domains, rated from 1 strongly disagree to 7 strongly agree The AGREE Next Steps Consortium. Appraisal of Guidelines for Research & Evaluation II, 2009.

AGREE II Domain 1: Scope and Purpose The overall objective(s) of the guideline is (are) specifically described Criteria: health intent(s) (ie, prevention, treatment, etc.), expected benefit or outcome, and target(s) (eg, patients population) 2. The health question(s) covered by the guideline is (are) specifically described 3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described

AGREE II Domain 2: Stakeholder involvement 4. The guideline development group includes individuals from all relevant professional groups 5. The views and preferences of the target population (patients, public, etc.) have been sought 6. The target users of the guideline are clearly defined

AGREE II Domain 3: Rigor of Development 7. Systematic methods were used to search for evidence 8. The criteria for selecting the evidence are clearly described 9. The strengths and limitations of the body of evidence are clearly described 10. The methods for formulating the recommendations are clearly described (eg, steps used in modified Delphi technique, voting procedures that were considered)

AGREE II Domain 3: Rigor of Development (cont.) 11. The health benefits, side effects, and risks have been considered in formulating the recommendations 12. There is an explicit link between the recommendations and the supporting evidence 13. The guideline has been externally reviewed by experts prior to its publication 14. A procedure for updating the guideline is provided

AGREE II Domain 4: Clarity of Presentation 15. The recommendations are specific and unambiguous 16. The different options for management of the condition or health issue are clearly presented (eg, description of population or clinical situation most appropriate to each option) 17. Key recommendations are easily identifiable

AGREE II Domain 5: Applicability 18. The guideline describes facilitators and barriers to its application 19. The guideline provides advice and/or tools on how the recommendations can be put into practice 20. The potential resource implications of applying the recommendations have been considered (eg, presenting cost information) 21. The guideline presents monitoring and/or auditing criteria (eg, criteria for assessing guideline implementation, adherence to recommendations, and impact)

AGREE II Domain 6: Editorial Independence 22. The views of the funding body have not influenced the content of the guideline 23. Competing interests of guideline development group members have been recorded and addressed

AGREE II Overall Guideline Assessment Rate the overall quality of this guideline from 1 lowest possible quality to 7 highest possible quality 2. I would recommend this guideline for use Yes Yes, with modifications No