1. Critical Appraisal of Clinical Practice Guidelines
Manit Srisurapanont, MD
Department of Psychiatry
Chiang Mai University
For Evidence-Based Medicine Workshop,
Chiang Mai, Nov 17-18, 2011

2. Outline Practice Guidelines
Assessing Recommendations (Gayatte et al. Users’ Guide to the Medical Literature: a Manual for Evidence-Based Medicine, 2008)
Appraisal of Guidelines for Research & Evaluation II (AGREE II, 2009)

3. Practice Guidelines
Are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances 
Play an important role in health policy formation
Practice guidelines are evidence-based if they undertake a review of the literature and link their concluding recommendations to the evidentiary base identified through the literature search 

4. From evidence to practice + clinical expertise**
*Haynes RB, Haines A. BMJ 1998;317:273-6.
**Haynes RB, et al. EBM 2002;7:36-8.

5. Assessing Recommendations: 1
1. Do the recommendations consider all relevant patient groups, management options, and possible outcomes?
All relevant patient groups, eg, low risk/high risk, more/less susceptible to adverse effects
All relevant management options, eg, surgical/medical , no-treatment option
All patient-important outcomes, eg, morbidity and mortality, quality of life, toxicity/adverse effects, cost to the patient or to society
Guyatt G, et al. User’s Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2008

6. Assessing Recommendations: 2
2. Are there systematic reviews of evidence that estimate the relative effect of management options on relevant outcomes?
For example, hormone replacement therapy: the incidence of hip fracture, breast cancer, endometrial caner, myocardial infarction, stroke, and dementia , as well as quality of life
The guideline developers’ systematic review must summarize the quality of the evidence on which they base their recommendations

7. Assessing Recommendations: 3
3. Is there an appropriate specification of values and preferences associated with outcomes?
It may be that the greater participation by methodologists, frontline clinicians, and members of the general public would lead to guidelines more in keeping with the public interest
It is important that authors of guidelines report the principal sources of judgments and the method of seeking consensus on values and preferences

8. Assessing Recommendations: 4
4. Do the authors indicate the strength of their recommendations?
The grades of recommendation, assessment, development, and evaluation (GRADE) system:
Strong vs. weak recommendation
High-, moderate-, low- and very-low-quality evidence
For example:
Strong recommendation, high-quality evidence: Benefits clearly outweigh risk and burden, or vice versa, RCTs without important limitations or overwhelming evidence from observational studies

9. AGREE II
The Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument was developed to address the issue of variability in guideline quality
The AGREE instrument is a tool that assesses the methodological rigor and transparency in which a guideline is developed
The original AGREE instrument (2001) has been refined, which has resulted in the new AGREE II (2010) and includes a new User’s Manual
AGREE II comprises 23 items in 6 domains, rated from 1 strongly disagree to 7 strongly agree
The AGREE Next Steps Consortium. Appraisal of Guidelines for Research & Evaluation II, 2009.

10. AGREE II Domain 1: Scope and Purpose
The overall objective(s) of the guideline is (are) specifically described
Criteria: health intent(s) (ie, prevention, treatment, etc.), expected benefit or outcome, and target(s) (eg, patients population)
2. The health question(s) covered by the guideline is (are) specifically described
3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described

11. AGREE II Domain 2: Stakeholder involvement
4. The guideline development group includes individuals from all relevant professional groups 5. The views and preferences of the target population (patients, public, etc.) have been sought 6. The target users of the guideline are clearly defined

12. AGREE II Domain 3: Rigor of Development
7. Systematic methods were used to search for evidence 8. The criteria for selecting the evidence are clearly described 9. The strengths and limitations of the body of evidence are clearly described 10. The methods for formulating the recommendations are clearly described (eg, steps used in modified Delphi technique, voting procedures that were considered)

13. AGREE II Domain 3: Rigor of Development (cont.)
11. The health benefits, side effects, and risks have been considered in formulating the recommendations
12. There is an explicit link between the recommendations and the supporting evidence
13. The guideline has been externally reviewed by experts prior to its publication
14. A procedure for updating the guideline is provided

14. AGREE II Domain 4: Clarity of Presentation
15. The recommendations are specific and unambiguous 16. The different options for management of the condition or health issue are clearly presented (eg, description of population or clinical situation most appropriate to each option) 17. Key recommendations are easily identifiable

15. AGREE II Domain 5: Applicability
18. The guideline describes facilitators and barriers to its application 19. The guideline provides advice and/or tools on how the recommendations can be put into practice 20. The potential resource implications of applying the recommendations have been considered (eg, presenting cost information) 21. The guideline presents monitoring and/or auditing criteria (eg, criteria for assessing guideline implementation, adherence to recommendations, and impact)

16. AGREE II Domain 6: Editorial Independence
22. The views of the funding body have not influenced the content of the guideline 23. Competing interests of guideline development group members have been recorded and addressed

17. AGREE II Overall Guideline Assessment
Rate the overall quality of this guideline
from 1 lowest possible quality to
7 highest possible quality
2. I would recommend this guideline for use
Yes
Yes, with modifications No