1 Introduction to Critical Appraisal Yulia Lin, MD, FRCPC Transfusion Medicine & Hematology Sunnybrook Health Sciences Centre University of Toronto TMR Journal Club, September 6, 2011

2 Why is Critical Appraisal important?

3 Vast amount of literature

4 New Studies in Transfusion

5 New Studies in Transfusion Past Studies in Transfusion

6 New Studies in Transfusion Past Studies in TransfusionStudies in other Sub-specialties

7 Why critical appraisal is important? Impossible to keep up with all of the literature Impossible to keep up with all of the literature Enables us to distinguish stronger evidence from weaker evidence Enables us to distinguish stronger evidence from weaker evidence Allows us to appropriately incorporate the evidence into our practice so as to improve patient care Allows us to appropriately incorporate the evidence into our practice so as to improve patient care

8 Transfusion Medicine Residency Objectives Transfusion Medicine Residency Objectives Scholar Scholar 2 - Evaluate medical information and its sources critically, and apply this appropriately to practice decisions 2 - Evaluate medical information and its sources critically, and apply this appropriately to practice decisions Describe the principles of critical appraisalDescribe the principles of critical appraisal Critically appraise retrieved evidence in order to address a clinical questionCritically appraise retrieved evidence in order to address a clinical question Integrate critical appraisal conclusions into clinical careIntegrate critical appraisal conclusions into clinical care Critically review published literature and apply appropriate utilization principles/practices /guidelinesCritically review published literature and apply appropriate utilization principles/practices /guidelines

9 Purpose of TMR Journal Club 1. To present a recent article in the transfusion literature 2. To learn how to critically appraise the medical literature

10 Steps to a successful JC 1. Choosing the article 2. Presenting the article 3. Critically appraising the article

11 Choosing an article… Where to find an article? Where to find an article? Recent table of contentsRecent table of contents NEJM, Lancet, JAMA NEJM, Lancet, JAMA Transfusion, Vox Sanguinis, Blood, etc. Transfusion, Vox Sanguinis, Blood, etc. Transfusion Medicine Reviews – Journal ClubTransfusion Medicine Reviews – Journal Club Transfusion Medicine CommunityTransfusion Medicine Community List of interesting articlesList of interesting articles

12 Choosing the article… Which article? Which article? RelevantRelevant Interesting to youInteresting to you What type of article? What type of article? Any articleAny article But some are easier than others…But some are easier than others…

13 Type of articles Randomized controlled trials – Therapy Randomized controlled trials – Therapy Observational studies – Harm Observational studies – Harm Systematic Reviews Systematic Reviews Guidelines Guidelines Diagnosis Diagnosis Prognosis Prognosis

14 First, a review…

15 Randomized Controlled Trial Population Randomization Intervention A Intervention B Outcomes measured

16 Randomized Controlled Trial Population Randomization Intervention A Intervention B Outcomes measured Which article?

17 Randomized Controlled Trial Randomization Fill in the blanks above: Population? Intervention? Outcome?

18 Randomized Controlled Trial Population Randomization Intervention A Intervention B Outcomes measured Which article?

19 Randomized Controlled Trial Randomization Fill in the blanks above: Population? Intervention? Outcome?

20 Observational Case-Control Population Cases (+Disease) Control (No disease) Exposure to Risk Factor Exposed Yes No Yes No

21 Observational Case-Control Population Cases (+Disease) Control (No disease) Exposure to Risk Factor Exposed Yes No Yes No Retrospective

22 Observational Case-Control Population Cases (+Disease) Control (No disease) Exposure to Risk Factor Exposed Yes No Yes No Which article?

23 Observational Case-Control Cases? Controls? Risk factor? Exposed Yes No Yes No Fill in the blanks: Population? Who are the cases and controls? What are the risk factors?

24 Observational Cohort Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No

25 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study

26 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Which article?

27 Cohort Studies Past PresentFuture Fill in the blanks: Who is the cohort? What is the risk factor? What is the outcome?

28 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Follow-up Cohort assembled Prospective Cohort Study

29 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Follow-up Cohort assembled Prospective Cohort Study Which article?

30 Cohort Studies Past PresentFuture Follow-up Cohort assembled Retrospective Cohort Study Fill in the blanks above: Who is the cohort? What is the risk factor? What is the outcome?

31 Systematic Review Overview Overview Summary of the medical literature that attempts to address a focused clinical questionSummary of the medical literature that attempts to address a focused clinical question Systematic review Systematic review Using methods designed to reduce the likelihood of biasUsing methods designed to reduce the likelihood of bias Meta-analysis Meta-analysis Review that uses quantitative methods to summarize the resultsReview that uses quantitative methods to summarize the results

32 Systematic Review Systematic review Systematic review Using methods designed to reduce the likelihood of biasUsing methods designed to reduce the likelihood of bias Meta-analysis Meta-analysis Review that uses quantitative methods to summarize the resultsReview that uses quantitative methods to summarize the results Meta-analysis of the randomized controlled trials of the hemostatic efficacy and capacity of pathogen-reduced platelets. Vamvakas EC. Transfusion May 2011;51:

33Guidelines “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” Attempt to address all issues and values relevant to a clinical decisionAttempt to address all issues and values relevant to a clinical decision Attempt to distill a large body of medical expertise into a convenient, readily usable formatAttempt to distill a large body of medical expertise into a convenient, readily usable format Make explicit recommendations with definite intent to influence what clinicians doMake explicit recommendations with definite intent to influence what clinicians do Field MJ. Clinical practice guidelines Hayward et al. JAMA 1995;274:570-4 Cook et al. Ann Int Med 1997;127:210-6

34Guidelines “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” Guidelines on oral anticoagulation with warfarin, 4 th edition. BCSH

35 How to do a critical appraisal?

36 Critical appraisal methods Most are based on Users’ Guides to the Medical Literature Most are based on Users’ Guides to the Medical Literature Available in a textbook form (2 nd ed)Available in a textbook form (2 nd ed) Brief version version For guidelines, different instruments For guidelines, different instruments AGREE instrument: centre/agree-ii/AGREE instrument: centre/agree-ii/ centre/agree-ii/ centre/agree-ii/

37

38 Steps to Critical Appraisal 1. Are the results valid? Have the results been influenced in a systematic fashion so as to lead to a false conclusion?Have the results been influenced in a systematic fashion so as to lead to a false conclusion? 2. What are the results? 3. How can I apply the results to patient care?

39

40

41RCT Population Randomization Intervention A Intervention B Outcomes measured

42 RCT - Are the results valid? Population Randomization Intervention A Intervention B Outcomes measured Was the assignment of patients to treatment randomized? Were all patients who entered the trial accounted for and attributed at its conclusion? Were patients, clinicians and study personnel kept "blind" to treatment received? Were the groups similar at the start of the trial? Aside from the experimental intervention were the groups treated equally?

43 RCT - Are the results valid? Randomization Adherence to transfusion strategies Was the assignment of patients to treatment randomized? Were all patients who entered the trial accounted for and attributed at its conclusion? Were patients, clinicians and study personnel kept "blind" to treatment received? Were the groups similar at the start of the trial? Aside from the experimental intervention were the groups treated equally? Elective cardiac Surgery N=50 Restrictive Hb <70 CPB Hb <75 postop Liberal Hb <95 CPB Hb <100 postop

44 RCT - What were the results? Population Randomization Intervention A Intervention B Outcomes measured How large was the treatment effect? How precise was the estimate of the treatment effect?

45 RCT - What were the results? Population Randomization Intervention A Intervention B Outcomes measured How large was the treatment effect? How precise was the estimate of the treatment effect?

46 RCT - How can I apply the results to patient care? Population Randomization Intervention A Intervention B Outcomes measured Can the results be applied to my patients?

47 RCT - How can I apply the results to patient care? Population Randomization Intervention A Intervention B Outcomes measured Can the results be applied to my patients? Were all clinically important outcomes considered? Are the likely treatment benefits worth the potential harm and costs?

48 Observational Cohort Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No

49 Cohort Studies Past PresentFuture Incidence of TRALI Pts receiving high plasma volume donations from LAPS II LAPS II: a retrospective cohort study of transfusion-related acute lung injury in recipients of high-plasma-volume human Leukocyte antigen antibody-positive or –negative components. Kleinman et al. Transfusion. Published online Mar 29, Exposure to HLA ab pos vs neg donations

50 Cohort - Are the results valid? Pts receiving high plasma volume donations from LAPS II Cohort without Disease Exposure to Risk Factor All hospital patients HLA ab pos HLA ab neg Disease TRALI No TRALI No Except for the exposure under study, were the compared groups similar to each other? Were the outcomes and exposures measured in the same ways in both groups? Was the follow-up of patients sufficiently long and complete? Is the temporal relationship correct? Is there a dose-response gradient?

51 Cohort - Are the results valid? Pts receiving high plasma volume donations from LAPS II Cohort without Disease Exposure to Risk Factor All hospital patients HLA ab pos HLA ab neg Disease TRALI No TRALI No Except for the exposure under study, were the compared groups similar to each other? Were the outcomes and exposures measured in the same ways in both groups? Was the follow-up of patients sufficiently long and complete? Is the temporal relationship correct? Is there a dose-response gradient?

52 Cohort - What are the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No How strong is the association between exposure and outcome? Consider magnitude and dose response? How precise is the estimate of the risk?

53 Cohort - What are the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No How strong is the association between exposure and outcome? Consider magnitude and dose response? How precise is the estimate of the risk?

54 Cohort - How can I apply the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No Are the results applicable to my patients?

55 Cohort - How can I apply the results? Cohort Cohort without Disease Exposure to Risk Factor Population Exposed Not exposed Disease Yes No Yes No Are the results applicable to my patients? What is the magnitude of the risk? Should I attempt to stop the exposure?

56 Putting it all together…

57 Tips for TMR Journal Club Choose the article carefully Choose the article carefully Include brief background for context Include brief background for context Present the article briefly (10 slides) Present the article briefly (10 slides) The main objective is the critical appraisal The main objective is the critical appraisal Internal validity:Internal validity: Focus on 1-2 key points Focus on 1-2 key points Explain one concept (bias, statistical test) Explain one concept (bias, statistical test) External validity: Take a standExternal validity: Take a stand Allow minutes for discussion Allow minutes for discussion