The Science of Guidelines The 7th ACCP Conference on Antithrombotic and Thrombolytic Therapy: Evidence-Based Guidelines Holger Schünemann, MD, PhD Italian National Cancer Institute, Rome, Italy McMaster University, Hamilton, Canada University at Buffalo, NY, USA

Topics for this talk  What makes guidelines evidence based in 2005?  High- vs low-quality evidence  Strong vs weak recommendations  Example recommendation  Example of the influence of values, preferences, and cost  Grading system

 Evidence – recommendation:transparent link  Explicit inclusion criteria  Comprehensive search  Standard consideration ofstudy quality  Conduct/use meta-analysis  Grade recommendations  Acknowledge values andpreferences underlyingrecommendations What makes guidelines evidence based in 2005? Schünemann J, et al. Chest. 2004; 126 Suppl 3:688S-696S.

Background  First ACCP guidelines in 1986 (J. Hirsh; J. Dalen)  Initially aimed at consensus  Group of experts and methodologists formally convening every 2 to 3 years  ~260,000 copies in 2001  7th conference held in 2003  87 panel members  22 chapters  Across subspecialities  Over 500 recommendations; 230 new  Evidence-based recommendations ACCP = American College of Chest Physicians.

Schünemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S.

The clinical question Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.  Transparent link: from evidence to recommendations  Explicit inclusion criteria MI = myocardial infarction; RCTs = randomized controlled trials; TIA = transient ischaemic attack.

Comprehensive search for evidence  Use questions to develop search strategy –e.g. identify all search terms (MeSH and keywords) for antiplatelet drugs or MI  Search –Cochrane Database of Systematic Reviews –Database of Abstracts of Reviews of Effectiveness –Cochrane Central Register of Controlled Trials –MEDLINE and EMBASE (1966 to December 2002) –ACP Journal Club  Provide search results –use EndNote ® software –e.g. 490 citations on thrombolysis in acute stroke ACP = American College of Physicians; MeSH = Medical Subject Headings.

Schünemann HJ et al. Chest 2004 Schunemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S

The ACCP grading system: GRADE* approach Clear separation of 2 issues:  Evidence: very low, low, moderate, or high quality? –methodological quality of evidence –likelihood of bias  Recommendation: weak or strong? –trade-off between benefits and downsides –patient values and preferences * GRADE = Grading of Recommendations Assessment, Development and Evaluation. GRADE Working Group. BMJ. 2004;328:

Why grade recommendations?  People draw conclusions about the –quality of evidence and strength of recommendations  Systematic and explicit approaches can help –protect against errors, resolve disagreements –communicate information  Change practitioner behaviour  Strong: apply uniformly –just do it  Weak: think about it –examine evidence yourself, consider patient circumstances very carefully and explore with the patient  However, wide variation in approaches (GRADE) GRADE Working Group. BMJ. 2004;328:

Grades of recommendation: methodological quality  High (A): consistent results from RCTs or observational studies with very strong association and secure generalization  Moderate (B): inconsistent results from RCTs or RCTs with methodological limitations  Low (C): unbiased observational studies (e.g. well-executed cohort studies)  Very low (D): other observational studies (e.g. case series) GRADE Working Group. BMJ. 2004;328:

RCT starts high – what moves quality down?  Flawed design and execution  Inconsistency  Indirectness  Imprecision  Reporting bias GRADE Working Group. BMJ. 2004;328:

Design and execution  Concealment  Intention-to-treat principle observed  Blinding  Completeness of follow-up  Early stopping GRADE Working Group. BMJ. 2004;328:

Moving quality up: observational studies – high or moderate quality?  Strong association –strong association: RR > 2 or RR < 0.5 –very strong association: RR > 5 or RR < 0.2  Dose–response relationship –bleeding risk associated with increasing INR (blood thinning with warfarin)  Plausible confounders would have reduced the effect INR = International Normalized Ratio; RR = relative risk. GRADE Working Group. BMJ. 2004;328:

Grades of recommendation: strength of recommendations  Stronger recommendations (we recommend) –high-quality methods with large, precise effect –benefits much greater than downsides, or downsides much greater than benefits –do it or don’t do it – we recommend –Grade 1  Weak recommendations (we suggest) –lower-quality methods with imprecise estimate –benefits not clearly greater or smaller than downsides –values and preferences very important –probably do it or probably don’t do it – we suggest –Grade 2

Example: stroke prevention In patients with history of non-cardioembolic stroke or TIA…, we recommend treatment with an antiplatelet agent (Grade 1A). Aspirin, aspirin + XR dipyridamole, or clopidogrel are all acceptable options for initial therapy. Clopidogrel: higher cost If we had to make a choice between aspirin and clopidogrel, what would that choice be? Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S. XR =extended release.

CAPRIE trial  Aspirin vs clopidogrel in patients at risk for cardiovascular event  19,185 patients, 3 subgroups with > 6,300 patients each (TIA/stroke; MI; peripheral arterial occlusive disease)  Mean duration of follow-up: 1.9 years  Primary outcome: ischaemic stroke, MI, or vascular death CAPRIE Steering Committee. Lancet. 1996;348: CAPRIE = Clopidogrel versus Aspirin in Patients at Risk of Ischaemic Events.

Clopidogrel better (Aspirin better) STROKEMI PAOD Total p = CAPRIE trial results: relative risk reduction CAPRIE Steering Committee. Lancet. 1996;348: PAOD = peripheral arterial occlusive disease.

CAPRIE trial results: absolute risk *p < 0.05 NNT 200 CAPRIE Steering Committee. Lancet. 1996;348: NNT = number needed to treat.

Which of the following recommendations should be given? 1. Aspirin over clopidogrel in patients with prior history of TIA/stroke? –OPTION 1 2. Clopidogrel over aspirin in patients with prior history of TIA/stroke? –OPTION 2

Audience at a prior thrombosis meeting

Values and preferences  Underlying values and preferences always present  Sometimes crucial  Important to make explicit

Judgements about recommendations 1. Benefit and downside evaluation Benefits << downsides Benefits ?  downsides Benefits ?  downsides Benefits >> downsides  ????  2. Recommendation (wording) STRONG Recommend don’t do it / should not do it WEAK Suggest probably don’t do it / might not do it WEAK Suggest probably do it / might do it STRONG Recommend do it / should do it

Example: stroke prevention  In patients with history of non-cardioembolic stroke or TIA…  …we recommend treatment with an antiplatelet agent (Grade 1A). Aspirin, aspirin + XR dipyridamole, or clopidogrel are all acceptable options for initial therapy  …, we suggest use of clopidogrel over aspirin (Grade 2B) Underlying values and preferences:  This recommendation places a relatively high value on a small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.

Judgement: benefits vs downsides*  (Quality of evidence)  Relative importance of the outcomes (benefits, harms, and burden)  Baseline risk of outcomes  Magnitude of the effect (RR)  Absolute benefit and harm  Precision of the estimates  Cost *Downsides include harm, burden, and cost

Guyatt G, et al. Chest. 2004;126 Suppl 3:179S-187S.

Summary  Guidelines require evidence-based methods  GRADE approach to grading  Integration of values and preferences  Grade 1: strong recommendation  Grade 2: weaker recommendation/suggestion  High transparency between evidence and recommendations

End

Disclosure Research funding: AstraZeneca, Pfizer, Amgen Honoraria/consultant fees  deposited in University at Buffalo or McMaster University research accounts: AstraZeneca, Boehringer Ingelheim, Pfizer, Amgen

Values and preferences  If available, should be integrated into recommendations and described by guideline developers  If unavailable, adequate representation of patients’ or society’s interests is assumed  To increase the likelihood of adequate representation, the process included review of recommendations by research methodologists, practicing generalists, and specialists

Schünemann HJ et al. Chest 2004

Grades of recommendation Guyatt G, et al. Chest. 2004;126 Suppl 3:179S-187S.

Grades of recommendation Guyatt G, et al. Chest. 2004;126 Suppl 3:179S-187S.