1. Teaching Evidence-Based Medicine Gary S Gronseth, MD, FAAN Professor of Neurology University of Kansas

2. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table Emphasize Evidence never enough –Apply to your patient –Incorporate patient values

3. A case… A 58 year-old right-handed man suddenly developed problems speaking, right lower facial weakness and right hand clumsiness. His symptoms slowly resolved over a week. He had a history of controlled HTN and no other risk factors. Head MRI: small left frontal infarct. EKG: sinus rhythm. MRA: no cranial artery stenosis. Echocardiogram: PFO

5. The Physician’s Dilemma To Close or Not to Close Even if the answer is unknown, a decision must be made!

6. Clinical Reasoning Close PFO? “Where I trained”

7. To Teach EBM… Explicitly Reason –Exclude the unreasonable Clinical Reasoning Close PFO?

8. Deceitful

9. “Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community.” “The consequences of a second stroke are potentially devastating. PFO closure is mandatory.”

10. Fallacious Irrelevant Rhetoric Psychological appeal Emotion-Driven Persuasion

11. Patient Intervention Co-intervention Outcome Determining relevance: Define the question

12. For patients with cryptogenic stroke and PFO does PFO closure vs no PFO closure reduce the risk of the next stroke Determining relevance: Define the question

13. Popular Appeal “Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community.”

14. Begging the Question “The consequences of a second stroke are potentially devastating. PFO closure is mandatory.”

15. Irrelevant Outcomes I’ll be sued. I’ll be reimbursed

16. Deceitful Fallacious

17. To Teach EBM… Explicitly Reason –Exclude the unreasonable Deceit Close PFO? Fallacy

18. Deceitful Fallacious Reasoned

19. Relevant Logical appeal Data-Driven Truth

20. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Principles Close PFO? Evidence Judgment

21. Principles Decision

22. Deductive Inference From Principles The left side of the brain controls the right side of the body My patient can’t control the right side of his body My patient has a problem with the left side of his brain

23. Principles Use a Parachute?

24. Principles Close PFO?

26. PFO Fibrous adhesions fail to seal the atrial septum after birth Persistence of a potential shunt between the right and left atria of the heart

27. PFO might allow paradoxical embolism Small emboli normally filtered by lung without clinically important consequence In patients with PFO, emboli can travel to the brain and cause ischemic stroke Closing the PFO will prevent future strokes

28. Principles Close PFO?

29. Reasoned Relevant Reason Logical appeal Data-Driven Truth Deduction (Principles)

30. Principles Close PFO? Evidence

31. Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. John had a stroke and PFO and was treated with closure, he didn’t have another stroke. Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. Analogy and Inductive Inference

32. Evidence What happened to patients?

33. Principles Close PFO? Evidence

34. Reasoned Relevant Reason Logical appeal Data-Driven Truth Induction (Evidence) Deduction (Principles)

35. Principles Close PFO? Evidence Judgment Best Guess Opinion Hypothesis

36. Reasoned Relevant Reason Logical appeal Data-Driven Truth Intuition (Judgment) Induction (Evidence) Deduction (Principles)

37. Distinguishing Opinion from Principles Is there equipoise? –Do reasonable people disagree? –Would an IRB approve a trial? –Is there an ongoing trial? Evidence separates judgment from principles Principles Close PFO? Evidence Judgment

38. Theory Scientific Method Experiment Hypothesis

39. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Principles Close PFO? Evidence Judgment

40. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy Strong Weak

41. Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. John had a stroke and PFO and was treated with closure, he didn’t have another stroke. Strong Weak Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke

42. Inferences from Evidence… Are not valid or invalid Are never certain Strong Weak

43. Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. John had a stroke and PFO and was treated with closure, he didn’t have another stroke. Strong Weak Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke Informally recalled cases Why is this a weak inference?

44. Inferences from informally recalled cases can mislead Too few cases Selective recall: remember those –That are more recent –With extreme results –That support our pre-conceptions Experts are not immune to these limitations

45. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error

46. Often too few cases Selective recall: remember those –That are more recent –With extreme results –That support our pre- conceptions Two Sources of Error Systematic Bias Random Chance

47. Find More Cases Retrospective Observational Sudy 2002 to 2010 Of all Stroke and PFO Cases: 319

49. Rats… I’m going to have to start counting these cases

50. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table

51. Relationships between variables PFO Closure and Stroke

52. 2 X 2 Table Outcome TreatmentNo stroke StrokeAll Closure167 No Closure152 Total23980319

53. Expected if No Relationship Outcome TreatmentNo stroke StrokeAll Closure12542167 No Closure11438152 Total23980319

54. Expected if no Realtionship Outcome TreatmentNo stroke StrokeAll Closure75%25%100% No Closure75%25%100% Total75%25%100%

55. “Actual” Outcome TreatmentNo stroke StrokeAll Closure15017167 No Closure8963152 Total23980319

56. “Actual” Outcome TreatmentNo stroke StrokeAll Closure90%10%100% No Closure59%41%100% Total75%25%100%

57. 2 X 2 Table Outcome TreatmentNo stroke StrokeAll Closureab167 No Closurecd152 Total23980319

58. Relative Risk stroke = b/(a+b) d/(c+d) Risk difference stroke = b/(a+b) - d/(c+d) Measures of Association Outcome TreatmentNo stroke StrokeAll Closureab167 No Closurecd152 Total23980319

59. Measure of Association Relative Risk Stroke Outcome Outcome TreatmentNo stroke Stroke Closure90%10% No Closure59%41% RR Stroke 10/41 = 0.24

60. Cryptogenic stroke patients receiving Closure were 0.24 times less likely to have stroke. Therefore, I should offer my patients with stroke and PFO Closure.

61. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table

62. Random (Sampling) Error --Incorrect result from bad luck Equally likely to be too high or too low Statistical power/precision --Measured by: P-values (p < 0.001) Confidence intervals RR 0.24: (95% confidence intervals 0.15 to 0.40)

63. Systematic Error Incorrect results from poor study design or execution More likely to be too high or too low Risk of Bias Measured: Semi-quantitatively Class of Evidence 0.50.7511.251.50.25 Truth Measured

64. Our Study +Cl -Cl Stroke No Stroke Patients not receiving Closure were more often older, diabetic and hypertensive Sometimes had to “guess” the outcome from the record.

65. Major Sources of Bias +Cl -Cl Poor Good Confounding Misclassification

66. Lower Risk of Bias The Randomized Masked Trial +Cl -Cl Poor Good R

67. Randomized Masked Trial Single Case Report What is the risk of Bias? Strong Weak

68. Find the best evidence Search online databases MEDLINE

71. There is insufficient evidence to support or refute the benefit or lack of harm of PFO closure. Conclusion Strong Weak

72. Despite the weak evidence, a decision must be made. Decide Strong Weak

73. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table Emphasize Evidence never enough –Apply to your patient –Incorporate patient values Principles Close PFO? Evidence Judgment

74. Induction is never certain Often the evidence is weak Even when strong, the Evidence never perfectly applies to your patient Explicitly consider how well the evidence applies to your patient

75. Incorporating patient values BenefitsRisks Uncertainty

76. Know what is not Known If you fail to acknowledge the uncertainty and tell the patient we know that the PFO should or should not be closed… You have failed to distinguish opinion from principles. Principles Close PFO Evidence Judgment

77. To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table Emphasize Evidence never enough –Apply to your patient –Incorporate patient values Principles Decision Evidence Judgment