1. 1 Introduction to Critical Appraisal Yulia Lin, MD, FRCPC Transfusion Medicine & Hematology Sunnybrook Health Sciences Centre University of Toronto TMR Journal Club, September 21, 2010

2. 2 Why is Critical Appraisal important?

3. 3 Vast amount of literature

4. 4 New Studies in Transfusion

5. 5 New Studies in Transfusion Past Studies in Transfusion

6. 6 New Studies in Transfusion Past Studies in TransfusionStudies in other Sub-specialties

7. 7 Why critical appraisal is important? Impossible to keep up with all of the literature Impossible to keep up with all of the literature Enables us to distinguish stronger evidence from weaker evidence Enables us to distinguish stronger evidence from weaker evidence Allows us to appropriately incorporate the evidence into our practice so as to improve patient care Allows us to appropriately incorporate the evidence into our practice so as to improve patient care

8. 8 Scholar Scholar 2 - Evaluate medical information and its sources critically, and apply this appropriately to practice decisions 2 - Evaluate medical information and its sources critically, and apply this appropriately to practice decisions Describe the principles of critical appraisalDescribe the principles of critical appraisal Critically appraise retrieved evidence in order to address a clinical questionCritically appraise retrieved evidence in order to address a clinical question Integrate critical appraisal conclusions into clinical careIntegrate critical appraisal conclusions into clinical care Critically review published literature and apply appropriate utilization principles/practices /guidelinesCritically review published literature and apply appropriate utilization principles/practices /guidelines

9. 9 Purpose of TMR Journal Club 1. To present a recent article in the transfusion literature 2. To learn how to critically appraise the medical literature

10. 10 Steps to a successful JC 1. Choosing the article 2. Presenting the article 3. Critically appraising the article

11. 11 Choosing an article… Where to find an article? Where to find an article? Recent table of contentsRecent table of contents NEJM, Lancet, JAMA NEJM, Lancet, JAMA Transfusion, Vox Sanguinis, Blood, etc. Transfusion, Vox Sanguinis, Blood, etc. Transfusion Medicine Reviews – Journal ClubTransfusion Medicine Reviews – Journal Club Transfusion Medicine CommunityTransfusion Medicine Community List of interesting articlesList of interesting articles

12. 12 Choosing the article… Which article? Which article? RelevantRelevant Interesting to youInteresting to you What type of article? What type of article? Any articleAny article But some are easier than others…But some are easier than others…

13. 13 Type of articles Randomized controlled trials – Therapy Randomized controlled trials – Therapy Observational studies – Harm Observational studies – Harm Systematic Reviews Systematic Reviews Guidelines Guidelines Diagnosis Diagnosis Prognosis Prognosis

14. 14 First, a review…

15. 15 Randomized Controlled Trial Population Randomization Intervention A Intervention B Outcomes measured

16. 16 Randomized Controlled Trial Population Randomization Intervention A Intervention B Outcomes measured Effects of tranexamic acid on death, vascular occlusive events, and blood transfusion in trauma patients with significant haemorrhage (CRASH-2): a randomised, placebo-controlled trial. CRASH-2 trial collaborators. Lancet. Published online June 15, 2010.

17. 17 Randomized Controlled Trial Trauma pts with Hemorrhage Randomization Placebo Tranexamic acid Death in hospital within 4 wks Effects of tranexamic acid on death, vascular occlusive events, and blood transfusion in trauma patients with significant haemorrhage (CRASH-2): a randomised, placebo-controlled trial. CRASH-2 trial collaborators. Lancet. Published online June 15, 2010.

18. 18 Observational Case-Control Population Cases (+Disease) Control (No disease) Exposure to Risk Factor Exposed Yes No Yes No

19. 19 Observational Case-Control Population Cases (+Disease) Control (No disease) Exposure to Risk Factor Exposed Yes No Yes No Retrospective

20. 20 Observational Case-Control Population Cases (+Disease) Control (No disease) Exposure to Risk Factor Exposed Yes No Yes No Prevalence of xenotropic murine leukaemia virus-related virus in patients with chronic fatigue syndrome in the Netherlands: retrospective analysis of samples from an established cohort. Van Kuppeveld et al. BMJ 2010;340:c1018.

21. 21 Observational Case-Control Population CFS (+Disease) Controls (No disease) Exposure to XMRV Exposed Yes No Yes No Prevalence of xenotropic murine leukaemia virus-related virus in patients with chronic fatigue syndrome in the Netherlands: retrospective analysis of samples from an established cohort. Van Kuppeveld et al. BMJ 2010;340:c1018.

22. 22 Observational Cohort Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No

23. 23 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study

24. 24 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Effect of duration of red blood cell storage on early and late mortality after CABG. van Straten et al. J Thor CV Surg 2010; published online.

25. 25 Cohort Studies Past PresentFuture Mortality before 30 days Mortality after 30 days CABG 1998-2007 Exposure to blood of different storage age Effect of duration of red blood cell storage on early and late mortality after CABG. van Straten et al. J Thor CV Surg 2010; published online.

26. 26 Cohort Studies Past PresentFuture In-hospital mortality Trauma 2000-2005 Nonmassive transfusion Plasma transfusion Effect of duration of red blood cell storage on early and late mortality after CABG. van Straten et al. J Thor CV Surg 2010; published online. Impact of plasma transfusion in trauma patients who do not require massive transfusion. Inaba et al. J Am Coll Surg June 2010;210:957-65.

27. 27 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Follow-up Cohort assembled Prospective Cohort Study

28. 28 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Follow-up Cohort assembled Prospective Cohort Study Transfusion practice and guidelines in Australian and New Zealand intensive care units Intensive Care Med 2010;36:1138-46.

29. 29 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Adherence to guidelines Patients receiving transfusion Transfusion Indications & triggers Transfusion practice and guidelines in Australian and New Zealand intensive care units Intensive Care Med 2010;36:1138-46.

30. 30 Systematic Review Overview Overview Summary of the medical literature that attempts to address a focused clinical questionSummary of the medical literature that attempts to address a focused clinical question Systematic review Systematic review Using methods designed to reduce the likelihood of biasUsing methods designed to reduce the likelihood of bias Meta-analysis Meta-analysis Review that uses quantitative methods to summarize the resultsReview that uses quantitative methods to summarize the results

31. 31 Systematic Review Systematic review Systematic review Using methods designed to reduce the likelihood of biasUsing methods designed to reduce the likelihood of bias Meta-analysis Meta-analysis Review that uses quantitative methods to summarize the resultsReview that uses quantitative methods to summarize the results Intravenous immunoglobulin for Guillain-Barre Syndrome. Hughes et al. Cochrane Database Syst Rev 2010 Jun 16;6:CD002063.

32. 32Guidelines “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” Attempt to address all issues and values relevant to a clinical decisionAttempt to address all issues and values relevant to a clinical decision Attempt to distill a large body of medical expertise into a convenient, readily usable formatAttempt to distill a large body of medical expertise into a convenient, readily usable format Make explicit recommendations with definite intent to influence what clinicians doMake explicit recommendations with definite intent to influence what clinicians do Field MJ. Clinical practice guidelines. 1990 Hayward et al. JAMA 1995;274:570-4 Cook et al. Ann Int Med 1997;127:210-6

33. 33Guidelines “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” Perioperative blood transfusion and blood conservation in cardiac surgery: the Society of Thoracic Surgeons and The Society of Cardiovascular Anesthesiologists clinical practice guideline. Ann Thorac Surg 2007 May;83(5Suppl):S27-86. Guidelines on the use of irradiated blood components. BCSH 2010. http://www.bcshguidelines.com/pdf/draft_irrad_0710.pdf

34. 34 How to do a critical appraisal?

35. 35 Critical appraisal methods Most are based on Users’ Guides to the Medical Literature Most are based on Users’ Guides to the Medical Literature Available in a textbook form (2 nd ed)Available in a textbook form (2 nd ed) Brief version http://www.ebem.org/usersguide.htmlBrief version http://www.ebem.org/usersguide.html http://www.ebem.org/usersguide.html For guidelines, different instruments For guidelines, different instruments AGREE instrument: http://www.agreetrust.org/resource- centre/agree-ii/AGREE instrument: http://www.agreetrust.org/resource- centre/agree-ii/ http://www.agreetrust.org/resource- centre/agree-ii/ http://www.agreetrust.org/resource- centre/agree-ii/

36. 36

37. 37 Steps to Critical Appraisal 1. Are the results valid? Have the results been influenced in a systematic fashion so as to lead to a false conclusion?Have the results been influenced in a systematic fashion so as to lead to a false conclusion? 2. What are the results? 3. How can I apply the results to patient care?

38. 38

39. 39

40. 40RCT Population Randomization Intervention A Intervention B Outcomes measured

41. 41 RCT - Are the results valid? Population Randomization Intervention A Intervention B Outcomes measured Was the assignment of patients to treatment randomized? Were all patients who entered the trial accounted for and attributed at its conclusion? Were patients, clinicians and study personnel kept "blind" to treatment received? Were the groups similar at the start of the trial? Aside from the experimental intervention were the groups treated equally?

42. 42 RCT - What were the results? Population Randomization Intervention A Intervention B Outcomes measured How large was the treatment effect? How precise was the estimate of the treatment effect?

43. 43 RCT - How can I apply the results to patient care? Population Randomization Intervention A Intervention B Outcomes measured Can the results be applied to my patients?

44. 44 RCT - How can I apply the results to patient care? Population Randomization Intervention A Intervention B Outcomes measured Can the results be applied to my patients? Were all clinically important outcomes considered? Are the likely treatment benefits worth the potential harm and costs?

45. 45 Observational Cohort Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No

46. 46 Cohort - Are the results valid? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No Except for the exposure under study, were the compared groups similar to each other? Were the outcomes and exposures measured in the same ways in both groups? Was the follow-up of patients sufficiently long and complete? Is the temporal relationship correct? Is there a dose-response gradient?

47. 47 Cohort - What are the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No How strong is the association between exposure and outcome? Consider magnitude and dose response? How precise is the estimate of the risk?

48. 48 Cohort - How can I apply the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No Are the results applicable to my patients?

49. 49 Cohort - How can I apply the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No Are the results applicable to my patients? What is the magnitude of the risk? Should I attempt to stop the exposure?

50. 50 Putting it all together…

51. 51 Tips for TMR Journal Club Choose the article carefully Choose the article carefully Include brief background for context Include brief background for context Present the article briefly (10 slides) Present the article briefly (10 slides) The main objective is the critical appraisal The main objective is the critical appraisal Internal validity:Internal validity: Focus on 1-2 key points Focus on 1-2 key points Try explaining a new concept (bias, statistical concept) Try explaining a new concept (bias, statistical concept) External validity: Take a standExternal validity: Take a stand Allow 20-30 minutes for discussion Allow 20-30 minutes for discussion