1. EVIDENCE BASED MEDICINE A new approach to clinical care and research Developed and presented by Judy Tarselli, RN Dubai, UAE Karachi, Pakistan October 2003 Organized by NKF cyberNephrology University of Alberta, Canada University of Alberta, Canadawww.cyberNephrology.org Special thanks to our sponsors Janssen-Cilag

2. PROGRAM OUTLINE 1. 1.Definition of EBM 2. 2.Basic Steps 3. 3.Trials, Studies and Reports 4. 4.Pros, Cons and Limitations 5. 5.EBM in Developing Countries 6. 6.EBM Library 7. 7.Advanced EBM

3. But first, a test…

4. A.Training, clinical experience and consultation with other professionals B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C.Preferences of the patient D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis Reports WHAT IS THE BASIS OF YOUR MEDICAL PRACTICE? (Check all that apply)

5. A.Training, clinical experience and consultation with other professionals B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C.Preferences of the patient D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis Reports WHAT IS THE BASIS OF YOUR MEDICAL PRACTICE? EXCELLLENT!

6. BUT… Past knowledge and practice might be outdated or inadequate Graduate Medical SchoolPracticed Physician

7. A.Training, clinical experience and consultation with other professionals B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C.Preferences of the patient D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reports WHAT IS THE BASIS OF YOUR MEDICAL PRACTICE? FANTASTIC!

8. BUT… This evidence may be biased, outdated, incorrect, or not applicable to your patient ARTICLESADVERTISEMENTS JOURNALS (1987 to present)

9. A.Training, clinical experience and consultation with other professionals B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C.Preferences of the patient D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reports WHAT IS THE BASIS OF YOUR MEDICAL PRACTICE? WONDERFUL! Mutual Respect + Shared Goals = Better Cooperation and Compliance

10. The patient should be involved in all important decisions But this is NOT always an easy task! And conflicts WILL occur!

11. But doctor, I DO want to have children! No salt? Lose weight? Forget it! Just give me a pill! I WON’T take that medicine… The side effects are INTOLERABLE! And conflicts WILL occur!

12. Education about current alternatives and risks is often needed… for both the Patient and the Doctor! But doctor, I DO want to have children! No salt? Lose weight? Forget it! Just give me a pill! I WON’T take that medicine… The side effects are INTOLERABLE!

13. I’ll discuss those risks with my husband. Yes, I’d like to try that new medication! Wow… I never knew that high blood pressure could be so dangerous at my age! Education about current alternatives and risks is often needed… for both the Patient and the Doctor!

14. The patient’s preferences MUST be considered! An important rule in Evidence Based Medicine… It STARTS with the patient and ENDS with the patient.

15. A. A.Training, clinical experience and consultation with other professionals B. B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C. C.Preferences of the patient D. D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reports WHAT IS THE BASIS OF YOUR MEDICAL PRACTICE? WOW!!! SUPERB!!!

16. In the practice of Evidence Based Medicine, it is the physician’s duty to find the best and most current information and apply it judiciously for the benefit of the patient.

17. But… A practice based exclusively on science and math is effective only if your patients are robots or clones! Don’t forget to allow for individual human differences and personal preferences!

18. A. A.Training, clinical experience and consultation with other professionals B. B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C. C.Preferences of the patient D. D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reports WHAT IS THE BASIS OF YOUR MEDICAL PRACTICE? If you checked all 4 items…

19. A. A.Training, clinical experience and consultation with other professionals B. B.Convincing evidence (non-experimental) from articles, case reports, product literature, etc. C. C.Preferences of the patient D. D.Active search of Randomized Controlled Trials, Systematic Reviews, Meta-Analysis reports You are practicing EVIDENCE BASED MEDICINE! CONGRATULATIONS!

20. EVIDENCE BASED MEDICINE A new approach to clinical care and research 1. 1.Definition of EBM 2. 2.Basic Steps 3. 3.Trials, Studies and Reports 4. 4.Pros, Cons and Limitations 5. 5.EBM in Developing Countries 6. 6.EBM Library 7. 7.Advanced EBM

21. “What is Evidence Based Medicine?” “And where did it come from?”

22. A BRIEF HISTORY 1980’s: McMasters University in Ontario, Canada Dr. David Sackett and colleagues proposed Evidence Based Medicine (EBM) as a new way of teaching, learning and practicing medicine. Dr. Sackett defines EBM as: “…The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”

23. Evidence Based Medicine It is a change in the way physicians practice medicine, teach and learn, and handle research. Clinical practice: Based on the best current evidence (not necessarily on how it’s always been done) Patient Care: Compassionate, patient-oriented (less authoritarian) Learning & Teaching: Problem-based, problem-solving more investigative, less know-it-all-by-yesterday Research: More stringent approach, better proof criteria (more demanding of proof, less room for error)

24. PATIENT PHYSICIAN INFORMATION Question or Problem THREE MAJOR COMPONENTS of EBM

25. PATIENT Values, Concerns Preferences, Expectations Life predicament PHYSICIAN Training & Experience Current Expertise Continued learning Demand for proof INFORMATION Clinically relevant Proven by research Best up-to-date evidence EBM THE ADDED DETAILS

26. PATIENT Values, Preferences Concerns, Expectations Life predicament PHYSICIAN Training Expertise Continued Learning Demand for proof EBM CHARITY EBM is not a required practice (yet) ENTHUSIASM Challenge, Variety, Change HUMILITY Non-authoritarian practice OPTIONAL COMPONENTS TO BE ADDED BY THE PHYSICIAN INFORMATION Clinically relevant Proven by research Current, up to date

27. “Isn’t this the way we have always practiced medicine?” “Aren’t these just the same old ingredients tossed into a new recipe?” When am I supposed to find the time to do that?

28. The basic steps of EBM

29. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

30. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

31. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

32. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

33. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

34. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

35. The Clinical Question The FIRST step The HARDEST step The MOST IMPORTANT step!

36. FACT: We all have informational needs! That is not a problem!

37. Problems arise if we fail to recognize those needs if we fail to recognize those needs if we fail to bridge the information gap if we fail to bridge the information gap if we fail to ask the right questions if we fail to ask the right questions

38. And also for others around you! Lee, exactly how much time did you spend on that big project? Hmmm… Is he about to give me a BONUS? Or is he about to FIRE me? It will make life easier for you... Asking good questions is a skill t Asking good questions is a skill to be learned.

39. A GOOD QUESTION… Is focused and relevant Is focused and relevant Provides clear communication Provides clear communication Clarifies your goal or need Clarifies your goal or need Will reduce the amount of time needed to obtain the answer Will reduce the amount of time needed to obtain the answer Lee, can you give me an accounting of the extra time you spent on that project so that I can charge it back to the client? Oh sure! I’ll have it on your desk by tomorrow!

40. Be specific Be specific Identify the problem, clarifiy the clinical issue Be answerable Be answerable through the literature Contain multiple aspects Contain multiple aspects (patient, options, comparisons, etc) WHEN PRACTICING EBM, a good question must also: ACTUAL CASE SCENARIO Large cauc male, age 40 2mo ago: Presented with classic nephrotic syndrome, significant symptoms. Bx showed IgAN. Cr 1.4, incr to 2 range, now 1.6 Tried prednisone 60mg qd - tolerated poorly w/tremors and depression. Needs new regimen, but others are aimed more at nephritic IgA rather than nephrotic syndrome. Suggestions? It should NOT involve a question of Personal Preference or Local Concern.

41. THE EVIDENCE BASED RESPONSE Posted on Nephrol 4/13/03 “In the study below, proteinuria and renal function improved on this combination: Ballardie FW, Roberts IS. Controlled prospective trial of prednisolone and cytotoxics in progressive IgAN. J Am Soc Nephrol 2002 Jan….” “I have patients on this regime who have benefitted.” Regards, Dr. Paulose P. Thomas Nephrologist - Belhoul Apollo Hospital, Dubai, UAE Respondant recommends cyclophosphamide and prednisolone (assuming secondary causes excluded) - a combination that allows for lower dose prednisolone…

42. BACKGROUND and FOREGROUND QUESTIONS (all part of EBM) FOREGROUND QUESTIONS BACKGROUND QUESTIONS NEW POSSIBILITIES INDEFINITE ANSWERS “Where do we want to go, and how else might we get there?” EXPERT GRADSTUDENT “Where are we now? And which way are we headed?” BASIC & CONCRETE

43. BACKGROUND QUESTIONS BASIC & CONCRETE 1. QUESTION Who, What, Where, When, Why, How 2. VERB is, causes, does, treats, reduces, cures, prevents, affects 3. GENERAL KNOWLEDGE ABOUT DISORDER clinical manifestations of disease, patient findings, differential diagnosis, etiology, patient experience, comorbid condition, screening and diagnostic tests, prognosis, therapy, risk factors, etc. EXPERT GRADSTUDENT

44. FOREGROUND QUESTIONS NEW POSSIBILITIES INDEFINITE ANSWERS PT AND/OR PROBLEM Differential dx, Unusual presentation, uncertain etiology, pt’s prior experience, comorbid conditions INTERVENTION Exposure, test. Prognostic factor, treatment, pt perception, etc. COMPARISON INTERVENTION OUTCOMES EXPERT GRADSTUDENT

45. EBM QUESTION: Should include multiple factors (Examples) PPATIENT PPATIENTtype of patient or population Ex: 47 yr male w/DM2 and cellulitis toe, 25 yr female w/DVT and chest pain EEXPOSURE EEXPOSUREenvironmental, personal, biological Ex: TB, tobacco, drug, diet, pregnancy or menopause, MRSA, allergy IINTERVENTION IINTERVENTIONclinical intervention Ex: medication, procedure, test, surgery, radiation, drug, vaccine CCOMPARISON CCOMPARISONcompare alternative treatment Ex: other prior, new or existing therapy OOUTCOME OOUTCOMEclinical outcome of interest Ex: Reduced death rate in 5 yrs, decreased infections, fewer hospitalizations

46. FRAMING THE QUESTION (Example: PICO) ELEMENTPROMPTS THE QUESTION: PatientHow would I describe a group of patients similar to mine? InterventionWhat main action am I considering? ComparisonWhat is/are the other options? OutcomeWhat do I (or the patient) want to happen (or not happen)? Example: P: In kids under age 12 with poorly controlled asthma on metered dose inhaled steroids… I: would the addition of salmetrol to the current therapy C:compared to increasing the dose of current steroid O:lead to better control of symptoms without increasing side effects?

47. CATEGORY OF QUESTION MAJOR CATEGORIES 1.Diagnosis 2.Prognosis 3.Therapy/ Treatment 3.Therapy/ TreatmentPICO 4.Harm (iatrogenic, other) 4.Harm (iatrogenic, other)PEO MISCELLANEOUS Quality of care Health economics Office Management Etc.

48. THE PATIENT’S QUESTIONS Must be considered! Often QUALITATIVE Often QUALITATIVE (not based on measureable outcomes) Feelings, ideas, experiences, preferences, concerns, fears, beliefs, ethnicity Usually based on LIMITED BACKGROUND Perception of problem Self-diagnosis Treatment wanted or needed Alternatives (read, heard, considered, tried) What is the patient hoping to avoid? What benefits does the patient want or need most? Etc.

49. QUANTITATIVE: “Solid Evidence” Measurable answer or response Measurable answer or response Necessary for scientific study Necessary for scientific study Necessary for the practice of EBM Necessary for the practice of EBM QUALITATIVE: “Quality of Life” “Fuzzy” data - Impact on daily life, work, family, etc. “Fuzzy” data - Impact on daily life, work, family, etc. May be very important and influential to decisions – especially for the patient May be very important and influential to decisions – especially for the patient Creates added challenge or twist to practice of EBM Creates added challenge or twist to practice of EBM QUANTITATIVE vs QUALITATIVE QUESTIONS

50. QUALY: QUALITY ADJUSTED LIFE YEAR

51. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

52. Find the Best Evidence “The Literary Search” HINT: If your desk looks like this, it’s probably the LAST place you should start looking!

53. Find the Best Evidence “The Literary Search” The BEST EVIDENCE is: External External - from outside resources (researchers, experts) Current Current – not out of date, most recent High Quality High Quality - accurate, precise, effective, safe Patient focused Patient focused - applicable and appropriate for your individual patient

54. FIVE STEPS TO FINDING THE BEST EVIDENCE 1. 1.IDENTIFY NEEDS:What type of information is needed? 2. 2.IDENTIFY RESOURCES: Types, Availability, Timeliness,Costs? 3. 3.SEARCH & RETRIEVE:Use efficient strategies 4. 4.REVIEW : Check quality and usefulness of info 5. 5.INTERPRET:Help patient understand info, application

55. WHAT TYPE OF INFORMATION IS NEEDED? WHAT CATEGORY IS THE QUESTION? Diagnosis Diagnosis Prognosis Prognosis Therapy Therapy Harm Harm

56. WHAT STUDY DESIGN FITS IT BEST? There are MANY study designs! EXPERIMENTAL TRIALS (Answers questions of diagnosis or treatment) Randomized Controlled Trials (RCTs) Controlled studies Blinded vs Open ETC. OBSERVATIONAL STUDIES Descriptive reports Retrospective studies Cohort studies Case Control ETC.

57. EXAMPLE Randomized Controlled Trials (RCT) “Gold Standard” of research Ideal experimental design - Best design for TREATMENT questions Must identify objective of treatment (Ex: cure, prevent complication, palliation, reassurance) Still not always the right intervention for individual patient at that particular time and place

58. What type of evidence best addresses the question, problem or issue? CLINICAL PRACTICEAPPROPRIATE DESIGN FOR CLINICAL RESEARCH Diagnosis, Dx testingCross-sectional study – not randomized trial PrognosisFollow-up studies of patients evaluated at same early point of illness Therapy, treatmentRCT or Systematic review of multiple RCTs must be used Avoid non-experimental approaches to avoid false conclusions about efficacy Exceptions: When treatment may be successful in an otherwise fatal condition When no studies are available (rare conditions, new treatments, etc.) HarmRCT, Cohort, Case-control OTHER INFORMATIONAL Explore hypothesisQualitative research History-takingCase control study Individual trial & errorn of 1 trial Following clinical courseCohort study RecordkeepingSystematic registry-based (computer supported) research Quality of Care researchIndividual peer review, Process Evaluation MISCELLANEOUS MISCELLANEOUSBasic Science, Genetics, Immunology, etc.

59. WHAT FORM OF INFORMATION? Case report Controlled Trial Systematic review Meta-analysis Clinical guidelines etc.

60. LITERARY SEARCH: NEXT STEP IDENTIFY YOUR RESOURCES Colleagues Consultation, Discussion (Caution: Response may be an outdated “This is what we do”) Paper resources books, reports, journals Electronic databases Health Literature Services specialized librarians, staff Review services, Abstract Services, etc.

61. SEARCH AND RETREIVE THE BEST EVIDENCE Learn and Practice various SEARCH STRATEGIES: To find useful information quickly To find useful information quickly To eliminate irrelevant, inappropriate or weak information To eliminate irrelevant, inappropriate or weak information Try to develop the habit of learning as you go; Not just in lengthy formal sessions!

62. LITERARY SEARCH STRATEGY ASK FOR HELP! SPECIALIZED PERSONNEL track down information, textbooks, articles, guidelinestrack down information, textbooks, articles, guidelines may provide electronic search support or trainingmay provide electronic search support or trainingEXAMPLES Medical Librarians Medical Informatics Specialists Specially trained staff member

63. LITERARY RESOURCES TEXTBOOKS (caution – most obsolete!) TEXTBOOKS (caution – most obsolete!) Traditional Traditional Evidence Based Evidence Based JOURNALS (may be outdated) JOURNALS (may be outdated) REVIEW ARTICLES (summaries, abstracts) REVIEW ARTICLES (summaries, abstracts) SYSTEMATIC REVIEWS (prepared in systematic, rigorous manner) Ex: Cochrane Collection SYSTEMATIC REVIEWS (prepared in systematic, rigorous manner) Ex: Cochrane Collection META-ANALYSIS META-ANALYSIS CLINICAL PRACTICE GUIDELINES CLINICAL PRACTICE GUIDELINES Summarized and easily digestible information

64. ELECTRONIC RESOURCES, DATABASES, INTERNET Bibliographic Database Example: Medline, PubMed Medical Information Services: Medscape, HDCN Review Services Subjective Systematic Reviews Meta-analysisExamples: Cochrane,Cochrane, Best Evidence,Best Evidence, Up to DateUp to Date

65. MORE GREAT INTERNET RESOURCES Websites cyberNephrology, National Kidney Foundation. NIDDK, American Heart Association, American Cancer Society. National Institutes of Health, etc Listserve Discussion Groups CyberNephrology, C-span, etc. Specialty Electronic Databases PsyclitCancerLitCINAHL (allied health and nursing journals) Etc

66. OTHER RESOURCES TapesVideosCD-ROMs Specialty seminars Product information and comparisons

67. A closer look at some Internet Resources…

68. MEDLINE WHAT IS IT? Searchable database of medical information compiled by National Library of Medicine in US 1966-present Catalogs articles from approx 4000 world journals (of estimated 12-15k total) SEARCH METHODS Any word or words (title, abstract, content, author name, institution, etc.) Medical Subject Heading (MeSH) terms A restricted thesaurus of medical titles Articles categorized by most specific possible MeSH heading

69. COST: FREE! Or may subscribe to companies with specialized search strategies: Ovid Technologies (ovid) Silver Platter Information (WinSPIRS)BENEFITS Free Vast databaseLIMITATIONS Not all articles are indexed on Medline (only 1/3 of approx 10 million!) Much material listed and described on Medline can only be accessed through journal article

70. MEDLINE: ELECTRONIC SEARCH STRATEGIES Search through “Clinical Queries” service of PubMed http://www.ncbi.nlm.nih.gov/clinical.html Medical Subject Headings (MeSH) Search filters Search by a text word can supplement a MeSH search Boolean search: “and”, “not”, etc. To increase sensitivity use “explode” command avoid using subheadings Online Tutorial is available!

71. COCHRANE LIBRARY Cochrane Database of Systematic Reviews - -systematically compiled reviews of intervention Cochrane Controlled Trials Register - -citations of controlled trials identified anywhere in the world Cochrane Review Methodology Database - -methodological papers relating to systematic reviews Etc.

72. BEST EVIDENCE Electronic version of two publications: Evidence Based Medicine American College of Physicians Journal Club Covers broad topics of information

73. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

74. CRITICAL APPRAISAL Interpreting the evidence How to read a paper How to read a paper How to do the math How to do the math

75. CRITICAL APPRAISAL IMPORTANT! You do NOT have to become a researcher, epidemiologist, or statistician to practice EBM. Focus on how to USE research reports – not on how to generate them!

76. HOWEVER… You must have a solid understanding of basic research principles and study designs in order to understand and interpret the evidence! CRITICAL APPRAISAL

77. TYPES OF STUDIES AND REPORTS Randomized Controlled Trial - “The Gold Standard” Systematic review Meta-analysis Retroactive vs Prospective Incidence Prevalence Case Control Cohort (Follow-up) Cross-sectional Ecologic Longitudinal Experimental Blinded vs Open Qualitative Screening

78. DETOUR

79. BASIC RESEARCH PRINCIPLES STUDY DESIGNS

80. THE TIME FACTOR When was the study done? In what time direction is it headed? What was its duration? RETROSPECTIVEPROSPECTIVE

81. THE TIME FACTOR When was the study done? What year? What technology? (ie: test, drug, equipment, procedure) Any associated social factor or historical event?

82. THE TIME FACTOR What was the Study Duration? Was it an appropriate length of time for the intended goal? Limited time study or ongoing? Was study completed? Stopped early?

83. “LOOKING BACK” Historical Review or Investigation “LOOKING FORWARD” Future Results The Great Unknown PRESENT PASTFUTURE In what direction is it headed? RETROSPECTIVEPROSPECTIVE

84. PRO May provide good direction for future studyMay provide good direction for future study “Hind Sight is 20/20” CON: Prone to BiasProne to Bias A“Fishing Expedition” for positive resultsA“Fishing Expedition” for positive results PRO Lower risk of biasLower risk of biasCON: May get faulty results based on incomplete data or insignificant subgroups (Example of Error: Untreated hypertension unlikely to cause cardiac event in child, so treatment is unnecessary below age 18yrs) PRESENT In what direction is it headed? RETROSPECTIVEPROSPECTIVE

85. “Was there a similar comparison group?”

86. Experimental Intervention No comparison group All subjects receive Experimental Intervention

87. Experimental Intervention NO EVENT OUTCOME EVENT “Trial and Error?”“Before & After?” “Trial and Error?” or “Before & After?” UNCONTROLLED STUDIES

88. PROBLEMS POSITIVE OUTCOME MAY BE DUE TO: Other factors Natural course of disease (some get better, some don’t!) Spontaneous change of health Placebo Effect Hawthorne Effect NEGATIVE OUTCOME May be due to study treatment. Could be disastrous! BENEFITS Can answer some questions about: likelihood of response adverse effect, etc. VERY PATIENT-SPECIFIC! MAY BE ONLY OPTION Rare conditions Previously unknown conditions “Trial and Error” “Before & After” UNCONTROLLED STUDIES Generally NOT accepted: Potentially Dangerous and Flawed Prone to BIAS! “Traditional Study Method” May produce strong results

89. SMALLPOX VACCINATION SMALLPOX VACCINE 1. 1796: Edward Jenner inoculates 8yr-old James Phipps with cowpox virus from a milkmaid’s hands. Child develops illness, recovers. 2. Two weeks later, inoculates same child with smallpox virus. Child survives, no illness. (Centuries later, smallpox eradicated!) n=1 GOOD! Resistant to Cowpox and Smallpox (NO DISEASE OUTCOME) James Phipps, age 8 years Example#1 UNCONTROLLED TRIALS: “TRIAL AND ERROR”

90. Drinks culture of H.pylori HELICOBACTER PYLORI - GASTRIC ULCERS 1982: Australian microbiologist Barry J. Marshall presents evidence showing a possible infectious cause for gastric ulcers. Suggests they may be treatable with antibiotics. Findings are met with disinterest and disbelief by medical community. Lacks support for further study. 5 years later: Prepares a broth of live organisms isolated from a gastric ulcer patient and drinks it. Becomes violently ill, develops severe acute gastritis. 1990’s Antibiotics are used routinely to cure some gastric ulcers! Example #2 NO OUTCOME SEVERE GASTRITIS n=1 UNCONTROLLED TRIALS: “TRIAL AND ERROR” Dr. Marshall Microbiologist

91. UNCONTROLLED TRIAL Experimental Intervention Present FUTURE May represent the ONLY treatment option for a new or rare disease DIED RECOVERED

92. Experimental Intervention Control Group STRONGLY PREFERRED! Reduces BIAS. Provides stronger results.

93. Nothing Placebo Observation only ExperimentalI ntervention Control group may receive… Only the TEST group receives the Experimental Intervention CONTROLLED STUDY Other IMPORTANT All other differences should be minimized or eliminated to reduce potential BIAS Gold Standard Treatment

94. Experimental Intervention Control Group RANDOMIZED CONTROLLED TRIAL (RCT) “The Gold Standard”

95. 1944 TUBERCULOSIS TREATMENT: Streptomycin vs Bedrest Streptomycin (n=50) Bedrest (n=50) THE FIRST RANDOMIZED CONTROLLED TRIAL By Sir Austin Bradford Hill (BLINDED)

96. Experimental Intervention Control Group OPEN vs BLINDED STUDIES OPEN

97. BLINDED TRIAL BLINDED OPEN vs BLINDED STUDIES

98. BLINDING SINGLE BLINDED: Pt unaware of what group s/he is in DOUBLE BLINDED: Pt and MD unaware OPEN LABEL: Everyone is aware

99. RANDOMIZED vs NON-RANDOMIZED TRIALS Experimental Intervention Control Group How is this group divided?

100. NON-RANDOMIZED Experimental Intervention Control Group Assigned to groups, usually by the researcher Potential for RESEARCHER BIAS!

101. RANDOMIZED Experimental Intervention Control Group Random method of assignment used Maximizes “sameness,” Eliminates BIAS!

102. RANDOMIZED CONTROLLED TRIAL (RCT) (EXPERIMENTAL TRIAL) Experimental Intervention Control Group Present FUTURE “The Gold Standard”

104. Experimental Intervention Trial of Medicine 1 Or placebo TRIAL SERIES FOR INDIVIDUAL PATIENT n=1 One patient, series of tests Experimental Intervention Trial of Medicine 2 Or placebo GOOD NO CHANGE OR BAD

105. Why a TRIAL SERIES for one patient? EXAMPLES: Trial of different medications and/or placebo for child reported to have ADHD symptoms that are not clinically apparent Trial of different analgesics for patient with chronic pain from a combination of diseases not previously studied PATIENT Must be blinded Must keep diary or complete questionnaire PHYSICIAN May need to be blinded (enlist help of pharmacist!) Must treat patient as usual in all other respects BENEFIT Produces data most applicable to the individual patient

106. Intervention A ONE GROUP, MULTIPLE TESTS CROSSOVER TRIALS Intervention B Intervention B Intervention A ASSESS OUTCOMES #1 ASSESS OUTCOMES #2 COMPARE OUTCOMES (Best if participants are blinded)

107. Intervention A CROSSOVER TRIALS Intervention B Intervention B Intervention A ASSESS OUTCOMES #1 ASSESS OUTCOMES #2 Fewer participants needed than a RCT! Lower costsAll are in experimental group PROS & CONS

108. Intervention A CROSSOVER TRIALS Intervention B Intervention B Intervention A ASSESS OUTCOMES #1 ASSESS OUTCOMES #2 MUST HAVE SHORT CARRYOVER EFFECT MUST HAVE SHORT WASHOUT EFFECT (OR WAIT A SUITABLY LONG WASHOUT TIME!) PROS & CONS

109. CASE CONTROL Present RISK FACTOR? (PAST) (“A LOOK BACK”)

110. CASE CONTROL Present NEVER SMOKED (PAST) (“A LOOK BACK”) SMOKER LUNG CANCER HEALTHY RISK FACTOR

111. CASE CONTROL Present NORMAL WEIGHT (“A LOOK BACK”) OBESITY DM TYPE II NON-DIABETIC RISK FACTOR

112. COHORT IS RISK FACTOR PRESENT? Future Outcome “FOLLOWUP DESIGN” (Exclude those with outcome already!)

113. COHORT IS RISK FACTOR PRESENT? Future Outcome “FOLLOWUP DESIGN” Present TO INVESTIGATE ETIOLOGY OR HYPOTHETICAL CAUSE OF DISEASE/OUTCOME

114. COHORT Present RISK FACTOR Hgb <9 EXAMPLE DIALYSIS PATIENTS Measures future outcome for dialysis pts w/o treatment of anemia

115. CROSS SECTIONAL DESIGN ? Cause ? Risk factors A look back

116. CROSS SECTIONAL DESIGN INFANT DEATHS SIDS DEATHS OTHER CAUSES RISK = SLEEP PRONE

117. NO CONTROL OVER CONTROL GROUP VARIATION IN TREATMENT OR METHOD NON-SIMILAR CONDITIONS Social Personal Comorbid conditions Other treatments Etc. Not usually accepted by medical journals (accepted in popular press, not reviewed) CURRENT GROUP OF PATIENTS Problems of looking back

118. RANDOMIZED & CONTROLLED TRIAL (RCT) Experimental Intervention Control Group PROSPECTIVE MAY BE BLINDED

119. START WITH YOUR TARGET POPULATION

120. Set CRITERIA for INCLUSION / EXCLUSION This will determine:  ELIGIBILITY at the start  VALIDITY at the end

121. START WITH YOUR TARGET POPULATION

122. ELIMINATE THOSE WHO DO NOT MEET THE CRITERIA

123. NEXT: GATHER A SAMPLE GROUP

124. THE SAMPLE GROUP WILL: Represent the target populationRepresent the target population Meet the criteria for inclusion / exclusionMeet the criteria for inclusion / exclusion SIDE NOTES… Study should be approved by an Ethics Committee Informed consent should be obtained from study participants

125. SAMPLE GROUP MAY BE SUBDIVIDED FURTHER STRATIFICATION Divide into subgroups based on important similar characteristics RANDOMIZATION Divide into sub-groups based on unknown confounders

126. STRATIFICATION “important similar characteristics” Examples: Male or Female Age Stage of illness Prior illness or treatment Hospital vs Office groups Comorbid condition Etc.

127. EXAMPLE OFSTRATIFICATION FEMALE MALE

128. RANDOMIZATION “unknown confounders” Examples: Postal code Postal code Month of birth Month of birth Random number Random number Etc. Etc.

129. EXAMPLE OFRANDOMIZATION DX IN JANUARY-JUNE DX IN JULY-DECEMBER

130. Experimental Intervention Control Group Divide your sample group(s) into STUDY GROUPS Next… Divide your sample group(s) into STUDY GROUPS “Test Group” “Baseline Group”

131. Experimental Intervention Control Group Divide your sample group(s) into STUDY GROUPS Next… Divide your sample group(s) into STUDY GROUPS Receives Experimental Intervention “Baseline Group” Nothing Nothing Observation Observation “Same” miscellaneous intervention (non- experimental) “Same” miscellaneous intervention (non- experimental) Placebo Placebo “Gold Standard” therapy - especially if unethical to do otherwise! “Gold Standard” therapy - especially if unethical to do otherwise! “Test Group”

132. ASSIGN PATIENTS TO STUDY GROUPS Experimental Intervention Control Group Use caution against bias! Sample GroupStudy Groups

133. Experimental Intervention Control Group STUDY INVESTIGATOR usually assigns patients to study groups. usually has a personal preference for the treatment or patient might unconsciously “work harder” to make the study work with non-preferred candidates = POTENTIAL FOR BIAS

134. Experimental Intervention Control Group Use random separation and assignment! RANDOMIZED CONTROLLED TRIAL (RCT)

135. Experimental Intervention Control Group RANDOMIZED CONTROLLED TRIAL (RCT)

136. Experimental Intervention Control Group FUTURE PresentProceed with study

137. Experimental Intervention Control Group RANDOMIZED CONTROLLED TRIAL (RCT) EXPERIMENTAL EVENT RATE (EER) CONTROL EVENT RATE (CER)

138. Experimental Intervention Control Group FUTURE Present RANDOMIZED CONTROLLED TRIAL (RCT) EXPERIMENTAL EVENT RATE (EER) CONTROL EVENT RATE (CER) “The Gold Standard”

139. Disadvantages of RCT Expensive large # pts needed Prolonged recruitment and follow-up time needed Funding difficult to obtain except w/support of pharmaceutical companies (problematic!)

140. RETURN FROM DETOUR

141. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

142. Interpreting the evidence How to read a paper How to read a paper How to do the math How to do the math CRITICAL APPRAISAL

143. EVALUATE WRITTEN EVIDENCE FOR… Quality Usefulness …BY ASSESSING Validity Reliability Relevance Clinical importance

144. Critical Appraisal: VALIDITY What was the original purpose of the study? When was it prepared? By whom? credentials? credentials? affiliations? affiliations? Sample population Did the subjects represent an appropriate test group? How were they selected? Were controls used? Were groups similar for important prognostic characteristics?

145. VALIDITY How was the information gathered and processed? Were groups treated equally except for trial therapy? Were appropriate criteria used to measure results? Were criteria applied rigorously? Was the study completed? (Or ended early for a specified reason?) Did the study account for all test subjects? Including subjects lost to follow-up? Were ALL pts analyzed in their allocated groups? (ie: INTENTION TO TREAT - not “completed treatment” analysis)

146. VALIDITYInformation Does the paper support its claims? Is the information accurately presented? Does it represent the truth?Results Are the results believable? To what degree of confidence? Ex: Disagreement is not uncommon on angiograms, EKGs, radiographs, pathology, PAP tests, etc.

147. VALIDITYComprehensiveness Size: Was it large enough to yield credible results? Thoroughness: Was it complete enough? Duration: Was it long enough?

148. CRITICAL APPRAISAL: CRITICAL APPRAISAL: RELIABILITY Do we trust the information and results? 1. 1.APPROPRIATE TYPE OF STUDY 2. 2.REPRODUCEABILITY 3. 3.INTERPRETATION OF RESULTS 4. 4.BIAS

149. RELIABILITY APPROPRIATE TYPE OF STUDY Was the type of study design used proper for the question? Example: RCT would be choice for questions on TREATMENT

150. RELIABILITY Are the Measurements and Results reproducibile? Different determinations may be caused by: Variation in measurement methods Different interpretation of results Lack of agreement Example: BP checks on same patient may vary. Are differences result of pt factor, examiner factor, treatment factor, normal variance Would the same results be obtained if patient is re-measured? (with identical procedure) at another time? by another person? Were any similar studies done? Was the information comparable? Did the results agree?

151. RELIABILITY INTERPRETATION OF RESULTS Is there consistency among researchers? Different determinations may be caused by: Variation in measurement methods Different interpretation of results Lack of agreement EXAMPLE: BANFF CONFERENCE - Setting standards in Transplant Pathology established by Kim Solez, MD Were any new questions or controversies raised by the study?

152. RELIABILITY IS THERE ANY EVIDENCE OF BIAS? A dangerous pitfall! PATIENTS PATIENTS RESEARCHERS RESEARCHERS

153. PATIENT BIAS Social Desirability Bias Patient responds in the way they perceive as correct to support MD to support MD to support a preconceived notion (ie: foods vs ADD) to support a preconceived notion (ie: foods vs ADD) Patient denies unhealthy behavior, gets misclassified Ex: Smoker vs Non-smoker

154. PATIENT BIAS Hawthorne Effect Authors must take steps to reduce this bias by treating all equally! Ex: Weigh all patients with same frequency, even for group not on special diet People act differently when they know they are being watched. Ex: Follow more careful diet when regular weigh-ins are scheduled

155. RESEARCHER BIAS Who sponsored or funded the study? Personal gain or loss from results? Affiliates Special interests Conflict of interest Biased goal? To satisfy editors and reviewers… rather than solve real life clinical problems

156. Criteria bias? Risk-avoidance by researchers (will focus energy on topics that produce positive results) Bias toward patients? Sample selection criteria used (inclusive, exclusive) Assignment to test group or control - Random? Blind? RESEARCHER BIAS

157. Data collection methods used applied similarly to all subjects, including controls? starting point – prospective/retrospective, stage of patient? Was assessment blind? Data analysis Were all potential subjects included in denominator or otherwise accounted? Were they evaluated in originally designated group? (INTENTION TO TREAT) RESEARCHER BIAS

158. REDUCING OR ELIMINATING BIAS AND ERROR CONDUCT BLIND STUDIES Single Double-blinded USE INDEPENDENT OBSERVERS When doctor and/or patient can not be blinded, blinded IO measures outcome IO may even be unaware of study hypothesis USE MULTIPLE OBSERVERS Ex: Send subject slides to multiple pathologists for interpretation ESTABLISH CLEAR STANDARDS Exact methods to use to reduce variation in technique among researchers Clear wording on surveys, etc VALIDATING INSTRUMENTS Repeat screening to check for correct answers on surveys More frequent evaluations or surveys prevent guesstimates common to less frequent evaluation

159. NEXT STEP IN CRITICAL APPRAISAL: RELEVANCE QUESTION: Is the report applicable to our… Problem? “Does it address the questions raised?” Patient(s)? “Will my patient respond like those in the study?” Practice? “Can it be done within my practice or circle?”

160. ARE THE STUDY PATIENTS Comparable within the study? (similar traits, age, socioeconomic group, stage of illness, treatment, etc.) Comparable to your patient? ARE THE STUDY PROFESSIONALS Comparable to you? (general/specialist, primary care/teaching hospital, etc.)

161. NEXT STEP in CRITICAL APPRAISAL CLINICAL IMPORTANCE Information can be true and interesting in theory, yet useless in clinical practice! 1.Is the information clinically important? 2.If yes, how important is it? study design - See: Hierarchy of Evidence study design - See: Hierarchy of Evidence weight of results weight of results

162. HEIERARCHY OF EVIDENCE (value of study design to maximize wt, minimize bias) 1. 1.Systematic Review of all relevant RCTs 2. 2.At least one properly designed RCT 3. 3.Trials and case studies 4. 4.Well-designed Controlled Trial without Randomization 5. 5.Well designed Cohort or Case Control Studies, preferably from >1 centre or group 6. 6.Multiple Time series with or without intervention 7. 7.(Exception: Dramatic results in uncontrolled trials, such as introduction of PCN in the 1940s) 8. 8.Opinions of respected authorities, based on 9. 9.Clinical expertise 10. 10.Descriptive studies 11. 11.Reports of Expert Committees

163. RANDOMIZED CONTROLLED TRIAL (RCT) Evaluation of RCT Were all clinically appropriate outcomes measured? Did an ethics committee approve the study? Any statistically significant results also clinically significant? Any significant adverse reactions? Was follow-up procedural analysis identical? Was continuous data analysis vs end of trial data used?

164. Interpreting the evidence How to read a paper How to read a paper How to do the math How to do the math

165. HOW TO DO THE MATH Incidence Prevalence Statistical Formulas +/- Predictive Values - Probability - The p value Relative Risk Risk Reduction Odds Ratios NNT (Number Needed to Treat) – Risk Reduction Confidence Intervals Sensitivity and Specificity Regression Analysis Subgroup Analysis Health Status Evaluation Health Economics

166. ACCOUNT FOR ALL even if Non-compliant Lost to follow-up Analyze as a member of the originally assigned group! Analysis SHOULD BE BASED ON INTENTION TO TREAT NOT on “completed treatment” analysis OUTCOMES: NOT “STUDY FAILURES” OUTCOMES RELATE TO EVERYDAY CLINICAL PRACTICE, including… Deaths Poor compliance Wrong treatment received Lost to follow-up Etc.

167. INCIDENCE & PREVALENCE NEPHROL, a service of NKF cyberNephrology 7/10/03 10:17:12AM Dear Nephrolers, I would like to know how to calculate incidence and prevalence of B and C virus in HD. Thank you in advance. Mario Cuba, MD Servicio de Nefrologia Hospital Lucia Iniguez Landin Holguin, Cuba

168. INCIDENCE & PREVALENCE Response from Michel Jadoul, MD NEPHROL, a service of NKF cyberNephrology Prevalence: total number of positive patients divided by total number of patients: 20+/200=10% Incidence: number of new positive cases/total number of cases negative at start of period (e.g; year)/period (year?) thus : for instance 2 new positive cases /100 negative cases at start of year=2%/year. M.Jadoul, M.D.

169. PREVALENCE B B B B B B B B B B 10 cases Hep B in 100 patients 10 / 100 = 0.10 PREVALENCE = 10%

170. INCIDENCE B B B B B B B B B B B B B B B 1 year period 90 HBsA(-) at start of study 5 new cases 5 / 90 = 0.06 INCIDENCE = 6% per year

171. B B B B B B B B B B B B B B B RECALCULATED PREVALENCE 16 cases Hep B in 100 patients 16 / 100 = 0.16 NEW PREVALENCE = 16%

172. COMPARISON STUDIES: NEW DIAGNOSTIC TESTS RESULTS OF GOLD STANDARD TEST EXPERIMENTAL TEST POSITIVE NEGATIVE DISEASE PRESENT NO DISEASE TRUE (+) a FALSE (+) b FALSE (-) c TRUE (-) d COMPARING A NEW TEST AGAINST THE GOLD STANDARD TEST

173. ACCURACY OF TEST - COMPARE TO GOLD STANDARD What is the usefulness of the test in various groups and subgroups of pts? RESULTS OF GOLD STANDARD TEST EXPERIMENTAL TEST TEST POSITIVE a + b TEST NEGATIVE c + d DISEASE PRESENT a + c NO DISEASE PRESENT b + d TRUE (+) a FALSE (+) b FALSE (-) c TRUE (-) d TESTS ARE RARELY 100% ACCURATE THEY MUST BE COMPARED AGAINST THE GOLD STANDARD

174. ACCURACY OF TEST - COMPARE TO GOLD STANDARD What is the usefulness of the test in various groups and subgroups of pts? RESULTS OF GOLD STANDARD TEST EXPERIMENTAL TEST TEST POSITIVE TEST NEGATIVE DISEASE PRESENT DISEASE NOT PRESENT TRUE (+) a FALSE (+) b FALSE (-) c TRUE (-) d TESTS ARE RARELY 100% ACCURATE THEY MUST BE COMPARED AGAINST THE GOLD STANDARD TOTALS c + d a + b TOTALS a+b+c+d b + da + c

175. ACCURACY OF TEST - COMPARE TO GOLD STANDARD What is the usefulness of the test in various groups and subgroups of pts? RESULTS OF GOLD STANDARD TEST EXPERIMENTAL TEST TEST POSITIVE TEST NEGATIVE DISEASE PRESENT DISEASE NOT PRESENT TRUE (+) a FALSE (+) b FALSE (-) c TRUE (-) d TESTS ARE RARELY 100% ACCURATE THEY MUST BE COMPARED AGAINST THE GOLD STANDARD TOTALS c + d a + b TOTALS a+b+c+d b + da + c SENSITIVITY SPECIFICITY

176. SENSITIVITY AND SPECIFICITY SENSITIVITY = PATIENT (+) TEST (+) Probability that patient WITH disease will have ABNORMAL result (instead of False Negative) SPECIFICITY = PATIENT (-) TEST (-) Probability that patient WITHOUT disease will have NORMAL result (instead of False Positive) OVERALL DISCRIMINATION OF TESTS High SENSITIVITY = low number false negatives High SPECIFICITY = low number of false positives Best accuracy if both factors are close to 100%

177. SENSITIVITY = a / (a + c) PATIENT (+) TEST (+) SPECIFICITY = d / (b + d) PATIENT (-) TEST (-) POSITIVE PREDICTIVE VALUE = a / (a + b) If pt tests (+), what is the likelihood s/he has the disease? NEGATIVE PREDICTIVE VALUE = d / (c + d) If pt tests (-), what is the likelihood s/he does NOT have the disease? PREVALENCE = (a + c) / (a + b + c + d) ACCURACY = (a + d) / (a + b + c + d) Proportion of results that correctly identify pts with and without disease (True + and True - as proportion of all results) LIKELIHOOD RATIO = sensitivity / (1 - specificity) How likely is it that + result accurately indicates disease, and - result no disease?

178. LIKELIHOOD RATIO (LR) Because Sensitivity and Specificity are NOT always 100% How likely is it that + result accurately indicates disease, and - result no disease? LIKELIHOOD RATIO FOR A POSITIVE RESULT (LR+) Probability of (+) result in diseased subject divided by Probability of (+) result in a healthy subject - or worded differently - Sensitivity divided by 100% - Specificity LIKELIHOOD RATIO FOR A NEGATIVE RESULT (LR-) 100% - Sensitivity divided by Specificity DISCRIMINATION = ZERO IF LR = 1

179. EVALUATING STUDY RESULTS Example: Mortality rates in 4444 pts x 5.4 trial years: 11.5% Placebo 8.2% Medicine RRR 29% (Relative Risk Reduction) ARR 3.3% (Absolute Risk Reduction) NNT 30 (Number needed to treat for 5.4 years to save 1 life

180. QALY = QUALITY ADJUSTED LIFE YEAR QUALITY RATING NOT specific for disease or treatment! Value rating - subject to different values of patients, physician, community Patient-based Economy-based - cost-utility/cost-effectiveness analysis Compares outcomes of conditions or intervention(s) State of health - vs -Time spent in it

181. QUALY: QUALITY ADJUSTED LIFE YEAR

182. TEST RESULTS: VARIABILITY FACT: Study results may vary. Group too small Not representative of larger group Etc. Age, sex, race, condition, culture, etc. Compliancy issues (patient and physician!) May be discovered or identified through study

183. TEST RESULTS: VARIABILITY FACT: Variability may or may not be significant Group too small Not representative of larger group Etc. Age, sex, race, condition, culture, etc. Compliancy issues (patient and physician!) May be discovered or identified through study

184. TEST RESULTS: VARIABILITY Group too small Not representative of larger group Etc. Age, sex, race, condition, culture, etc. Compliancy issues (patient and physician!) May be discovered or identified through study Obviously faulty studies should be eliminated.

185. TEST RESULTS: VARIABILITY Group too small Not representative of larger group Etc. Age, sex, race, condition, culture, etc. Compliancy issues (patient and physician!) May be discovered or identified through study Some variability should be expected in the rest.

186. TEST RESULTS: VARIABILITY Group too small Not representative of larger group Etc. Age, sex, race, condition, culture, etc. Compliancy issues (patient and physician!) May be discovered or identified through study Some factors are completely unexpected.

187. and variability due to The statistics allow us to distinguish between

188. PROBABILITY CHANCE WARNING PROBABILITY should NOT be confused with CHANCE! Statistically significant Results are measurable and predictable Affected by sample size (1 in 20 is less convincing than 1 in 10,000) No statistical significance Random, unpredictable

189. A Study in Probability… QUESTION #1: What percentage of patients will develop diarrhea while taking Antibiotic A? QUESTION #2: Will the results be the same, better or worse on Antibiotic B? PROBABILITY

190. 50 20 study groups 50 pts each study 1000 patients total QUESTION #1: What percentage of patients will develop diarrhea while taking Antibiotic A?

191. 46108121416% Conduct studies Organize results NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA QUESTION #1: What percentage of patients will develop diarrhea while taking Antibiotic A?

192. 46108121416% NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA IDENTIFY STATISTICALLY SIGNIFICANT RESULTS The Bell Curve

193. COMMON Most COMMON result = 46108121416% MODE NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA

194. CENTER The CENTER of distribution = 4%68121416% MEDIAN NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA 10

195. MODE COMMON10% MODE (most COMMON result) = 10% MEDIANCENTER10% MEDIAN (the CENTER of distribution) = 10% NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA DETERMINE RESULTS OF STUDY 46108121416%

196. “10% of patients will develop diarrhea while taking Antibiotic A.” NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA 10 468121416% CONCLUSION

197. 468121416 NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA Wait… What about the other 15 study groups? You can’t just ignore them! 10 Or can you?

198. 468121416 PERCENTAGE OF PATIENTS WITH DIARRHEA Wait… What about the other 15 study groups? You can’t just ignore them! 10 Or can you? 15 groups x 50 per group = 750 patients (75%!)

199. 468121416 PERCENTAGE OF PATIENTS WITH DIARRHEA Wait… What about the other 15 study groups? You can’t just ignore them! 10 Or can you? Studies must account for ALL patients

200. 468121416 NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA Wait… What about the other 15 study groups? You can’t just ignore them! 10 Or can you? Results should not be ignored, but STATISTICAL SIGNIFICANCE may be questioned.

201. NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA 468121416 10 My head is starting to feel heavy… So, how is STATISTICAL SIGNIFICANCE determined?

202. PROBABILITY is determined by it. CHANCE is not related to it at all! STATISTICAL SIGNIFICANCE How is it determined?

203. PERCENTAGE OF PATIENTS WITH DIARRHEA Let’s use our study on “Antibiotic A” as the example 468121416% 10 NUMBER OF STUDIES 20 groups were tested. Only 1 of 20 groups landed at each end of the Bell Curve…. WHY? SAMPLE VARIATION? or CHANCE?

204. PERCENTAGE OF PATIENTS WITH DIARRHEA “There is a 1 in 20 chance that other patients will land in these categories.” But that would NOT be a correct statement! It is tempting to say, 468121416% 10 NUMBER OF STUDIES

205. PERCENTAGE OF PATIENTS WITH DIARRHEA “There is a 1 in 20 chance that other patients will land in these categories.” Why? Because CHANCE can not be used to predict future results! It is tempting to say, 468121416% 10 NUMBER OF STUDIES

206. CHANCE is based on RANDOM possibility. Example: THE COIN TOSS Coins tossed: 20 “Heads”17 (85%) “Tails” 3 (15%) Statistical Significance : ZERO! The next coin toss will still produce a random result! Random results can not be used to calculate Statistical Probability.

207. 4%68121416%10 “There is a 1 in 20 chance that patients will land in one of these two categories.” So instead of measuring “chance”… PERCENTAGE OF PATIENTS WITH DIARRHEA 1 in 20 chance

208. 4%68121416%10 We need to determine the PROBABILITY! 1 in 20 chance Translates into 5% probability PERCENTAGE OF PATIENTS WITH DIARRHEA “There is a 5% probability that patients will land in one of these two categories.”

209. 4%68121416% There is a 5% probability that 4% of patients will develop diarrhea on Antibiotic A. There is a 5% probability that 16%of patients will develop diarrhea on Antibiotic A. 10 PERCENTAGE OF PATIENTS WITH DIARRHEA The results now look like this:

210. 4%68121416%10 1 in 20 chance Probability = 5% PERCENTAGE OF PATIENTS WITH DIARRHEA FRACTION PERCENTAGE And now… Let’s abbreviate it some more! p = 0.05 Translates to DECIMEL

211. 4%68121416%10 1 in 20 chance PERCENTAGE OF PATIENTS WITH DIARRHEA FRACTION PERCENTAGE p = 0.05 Translates to DECIMEL …by changing “% probability” to the “p value” Probability = 5%

212. 4%68121416%10 1 in 20 chance PERCENTAGE OF PATIENTS WITH DIARRHEA p = 0.05 Translates to FRACTION DECIMEL PERCENTAGE The “p value” is statistically important! Probability = 5%

213. 4%68121416%10 1 in 20 “chance” p = 0.05 It determines statistical PROBABILITY. “p value” Probability = 5%

214. 4%68121416%10 PROBABILITY vs CHANCE …bPROBABILITY …but PROBABILITY is very important! It tells us the likelihood that something will happen. CHANCE So, CHANCE has ZERO significance PERCENTAGE OF PATIENTS WITH DIARRHEA NUMBER OF STUDIES

215. “There is a 5% probability that our study patients will fall into either of these categories.” OUR PROBABILITY STATEMENT p = 0.05 PERCENTAGE OF PATIENTS WITH DIARRHEA 46108121416%

216. Anything less than 5% (p= 0.05) MAY be due to chance. THAT IS A LOW PROBABILITY! p = 0.05 PERCENTAGE OF PATIENTS WITH DIARRHEA 46108121416%

217. 46108121416% p = 0.01 And anything less than 1% (p= 0.01) is MOST LIKELY due to chance! THAT IS A LOW PROBABILITY!

218. (p= 0.05) Anything less than 5% (p= 0.05) MAY be due to chance. THAT IS A LOW PROBABILITY! 46108121416% p = 0.01 (p= 0.01) Anything less than 1% (p= 0.01) is MOST LIKELY due to chance. p = 0.05

219. (p= 0.05)becomes VERY SIGNIFICANT (p= 0.05) becomes VERY SIGNIFICANT But when compared to another study… 46108121416 And (p= 0.01) becomes HIGHLY SIGNIFICANT! 18%

220. (p= 0.05)becomes VERY SIGNIFICANT (p= 0.05) becomes VERY SIGNIFICANT But when compared to another study… 46108121416 And (p= 0.01) becomes HIGHLY SIGNIFICANT! 18%

221. 4%6108121416 Antibiotic A NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA 18%

222. 4%6108121416 Antibiotic B NUMBER OF STUDIES PERCENTAGE OF PATIENTS WITH DIARRHEA 18%

223. (p= 0.05)becomes VERY SIGNIFICANT (p= 0.05) becomes VERY SIGNIFICANT DIFFERENCE = STATISTICALLY SIGNIFICANT 46108121416 And (p= 0.01) becomes HIGHLY SIGNIFICANT! 18%

224. The P value Measures Probability How often is this finding expected to occur? Determines Statistical Significance What is the likelihood these findings are TRUE or FALSE? Do the comparative findings show a significant difference? Meaningful ranges p >0.05 Not significant p <0.05 Statistically SIGNIFICANT p <0.01HIGHLY SIGNIFICANT! Does probability provide PROOF? NO! We could be misled by it. The sample size is very important when determining probability!

225. SAMPLE SIZE and PROBABILITY EXAMPLE: 100 pieces of fruit are in a bin: APPLES and ORANGES You close your eyes and pick 10 of them: Question: Does your sample accurately represent what is in the bin?

226. Answer: No! Larger samples provide a closer approximation of the populations they represent... But the only way to get 100% proof is to examine “all of the fruit in the bin!”

227. The P value Meaningful ranges p >0.05 Not significant p <0.05 Statistically SIGNIFICANT* p <0.01HIGHLY SIGNIFICANT!** *Significance only means that CHANCE is an unlikely explanation for the resultsLIMITATION The p value determines LIKELIHOOD… Not proof! CAUTION Statistical significance does not necessarily imply any clinical significance! EXAMPLE: Looking through a pinhole will improve vision in most people… But would this be an appropriate treatment for your myopic patients? (Key Topics in EBM )

228. MISCELLANEOUS STUFF

229. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

231. PATIENT PHYSICIAN INFORMATION Question or Problem (The Three Major Components of EBM)

232. PATIENT PHYSICIAN INFORMATION

233. PATIENT

234. PATIENT “A METHODOLOGICAL MINEFIELD”

235. PATIENT “A METHODOLOGICAL MINEFIELD” INFORMATION Difficult time understanding background information PHYSICIAN Personal priorities may conflict with yours

236. PATIENT Recognize: Needs Choices Preferences Values Socioeconomic concerns INFORMATION Help the patient to understand and interpret available information PHYSICIAN Respect the personal priorities of the patient Help the patient to negotiate a decision on intervention, treatment CONFLICT? SEPARATE THE ISSUES!

237. PATIENT INFORMATIONPHYSICIAN PATIENT And then help the patient pull it all together again

238. KEY POINTS PARADIGM SHIFT OLD: Doctor had authority (despite the pile of unread journals!) NEW: Current Best Evidence leads medical practice but it MUST be individually applied THE INDIVIDUAL PATIENT Every patient is different. Treat YOURS and not others The “ideal” course of action is not necessarily best for THIS patient. EBM + Psychosocial factors = THIS patient should be advised to take THIS therapy at THIS point in time.

239. THE FIVE BASIC STEPS OF EBM 1. 1.Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process

240. INFORMATION Adequate resources? Ease or Difficulty of finding and getting desired information? Costs? INTERVENTION Patient response or acceptance? Ease or Difficulty of Application? Clinical outcomes? EBM PROCESS EFFECT ON PRACTICE Will this particular experience change our thinking or practice? SELF EVALUATION How did we do? (Question, Search, Appraise, Apply) How could we improve our own EBM performance? Evaluate

241. EBM: PROS, CONS and LIMITATIONS

242. PROS Clinicians update knowledge base routinely Improved understanding of research methods Physician becomes more critical in use of data Increased confidence in management decisions Increased computer literacy, data search technology Better reading habits Provides framework for group problem solving, team generated practice

243. Transforms weakness or paucity of knowledge into positive change OK to be uncertain OK to be skeptical OK to be flexible Integrates medical education, research and clinical expertise Can be learned by non-clinicians – other HCWs, patient groups, purchasers, etc. Allows us to keep up with our better-educated patients!

244. Increased contribution of junior MDs Increased patient benefit Better communication with patients re: rationale of management decisions Promotes better and more appropriate use of limited resources May reduce costs or medical care or practice by eliminating outdated or unnecessary factors Can be learned at any stage of physician’s career

245. CONS Time consuming Information overload Time to learn and practice Time may be needed for team conferencing, planning and review Takes $$$ to establish resource infrastructure – library, office, etc. computers, peripherals

246. Internet costs Programs, software information, CD-ROMS Subscription costs – online and paper resources May increase cost of care (but hopefully offset by elimination of unnecessary medical interventions, tests, journals, etc. – plus save time in getting proper intervention) Online references made to unavailable journals or references Exposes gaps in the evidence (but provides ideas for researchers!)

247. Requires computer skills (but can be done with minimal computer literacy and skill) May expose your current practice as obsolete or dangerous (loss of authority and respect)

248. LIMITATIONS Lack of evidence (shortage of studies) Difficulty applying evidence to care of a particular patient Barriers to the practice of high quality medicine Lack of skills (search, appraise, etc.) (foster development of new skills!) Lack of time to learn and practice EBM(promotes lifelong learning thru better focus) Lack of physician resources for instant access to evidence (EBM has worldwide applicability)

249. RESTRICTED AVAILABILITY OF LAB TESTS NON-TEXTBOOK CASE co morbidity, additional risk factors AFFORDABILITY (MD & PT)“I can’t afford to practice EBM.” Language barriers – available evidence may be unreadable, should be included

250. Physician attitude: Can be the greatest limitation! “It decreases the importance of my clinical expertise” (that’s a necessary component!) “It only applies to those involved in research.” (promotes cooperation among multiple physicians) “It ignores patient values and preferences.” “It’s just another cookbook approach to medicine.” “It’s a poorly disguised way to cut medical costs.” (cost of care may actually increase) “It’s a way to ration care and resources.” (Provides better utilization of avail resources)

251. DISAGREEMENT Pt’s comfort, choice, acceptance, values preferences Vs MD’s recommendations DOES RISK OR SIDE EFFECTS OF TREATMENT OUTWEIGHT THE BENEFITS?

252. The unanswered question… “DOES EBM REALLY MAKE A DIFFERENCE?” Effect of practicing EBM on patient outcome is actually unknown – no studies done EBM good based on population studies: (ie: Pts who rec’d ___ generally fare better than those who don’t)

254. EBM IN DEVELOPING COUNTRIES LIMITED RESOURCES May help to eliminate unnecessary or poor quality screening tests (ie: resting EKG to screen for CAD = high false negative and false positive rates) LIMITED DRUG REGULATION Approval for drug marketing easy - promotes insurgence of new drugs for questionable indications, limited effectiveness, false claims, inflated prices based on ad response (include “more expensive is better”)

255. EBM IN DEVELOPING COUNTRIES LIMITED CAPACITY FOR CME Drug companies - may sponsor meetings that are little more than captive marketing sessions or biased education sessions (drug education vs promo) Result may be push for more expensive, less effective treatments (ie push for CCB’s over BB’s) - calc channel blockers over Beta Blockers

256. EBM IN DEVELOPING COUNTRIES LIMITED ACCESS TO LITERATURE DATABASES Desktop computer with CD ROM reader and modem ($900) Electricity 1 yr subscription to MedLine on CD ROM (?500) Internet connection $25/mt Convince administrators of expense: Publicly cite how searches help with lectures, research and patient care management decisions Get equipment from drug companies (usually strings attached)

257. EBM IN DEVELOPING COUNTRIES LIMITED ACCESS TO ADEQUATE LIBRARY FACIILITIES ALMOST INEVITABLE IN DEVELOPING COUNTRIES Identify resources via search, but then unable to retrieve articles! A top EBM practitioner (Philippines) recommends: 1. 1.Top 3 medical libraries in your country 2. 2.Multinational drug company libraries 3. 3.Friends and colleagues - including in other countries

258. EBM IN DEVELOPING COUNTRIES QUESTIONABLE APPLICABILITY OF ARTICLES RETRIEVED Article describes a treatment that worked in one country, but seems impossible in yours Check… Are there pathophysiologic differences? Will patient differences diminish the treatment response? Patient compliance issues? Provider compliance issues? Co-morbid conditions which will alter the benefits or risks?

259. EBM IN DEVELOPING COUNTRIES OBSTACLES TO TEACHING OR LEARNING EBM Your Hospital or Institution does not reimburse for time spent on Continuing Medical Education programs The standard 5-day workshop would be far too costly to provide or attend! Need to learn the basics - computer skills, etc. TRY THESE! Combine efforts to learn more and practice EBM with handful of colleagues (small group learning) Ask about basis for information provided by drug reps, medical supply companies, etc. It will prompt them to provide you with on the spot teaching and better information, too!

260. EBM LIBRARY BASIC REQUIREMENTS Convenient – easy access at point of contact with patient if possible Current – Up to date information Electronic Database – Should be included Online CD-ROM

261. ELECTRONIC DATABASES Evidence-Based Medicine Reviews (EBMR) – from Ovid (ovid.com) - combines Cochrane, Best evidence, Evidence Based Mental health, EB Nursing, Cancerlit, healthstar, AIDSline, Medline, and journal links (Described by one EBM specialist as “the best”) Cochrane Library – “Gold Standard” for systematic reviews Best Evidence Medline – world’s largest, free resource – over 10 million references

262. PERSONNEL Medical Librarian Informatics Specialist “We can learn a great deal about current best information sources from librarians and other experts in medical informatics, and should seek hands-on training from them as an essential part of our clinical training.” (ch 2 p29-30 – Blue circled 2)

263. PRINTED RESOURCES TEXTBOOKS most obsolete! Some updated yearly, plus heavy references and scientific evidence for support Clinical Evidence (BMJ Publishing Group & ACP – 1999- present) Evidence-Based On Call (http//cebm.jr.ox.uk/eboc/eboc.html) Up To Date (General medicine, CD format, Medline abstracts used for evidence) Scientific American Medicine – limited references from Medline, Harrisons

264. JOURNALS Traditional Journals subject to author submissions specialists need to read and evaluate may subscribe to services that send articles of interest to your specialty timely, instant information at time of publication Ex: NEJM, Clinical Nephrology, etc. Evidence Based journals selects best studies from multiple journals of interest, summarizes best evidence Good for use by generalists Lag time from original publication: 3-6 months Ex: Evidence Based Medicine, Evidence Based Nursing, Evidence Based CV Medicine, etc.

265. SPECIAL RESOURCES WHO Blue trunk Hinari PATIENT RESOURCES Medical treatmentswww.nlm.nih.gov/medlinepluswww.nlm.nih.gov/medlineplus Medical guidelineswww.guideline.govwww.guideline.gov

266. THE NEXT LEVEL: ADVANCED EBM SUMMARIZE AND STORE INFORMATION Future reference SHARING INFO Local Colleagues Author paper TEACHING New skills or treatments EBM practices OTHER APPLICATIONS Care of the individual patient Team protocols Hospital or practice guidelines

267. EBM in Medical Education Message to medical educators from Trisha Greenhalgh, MD, co-author of Evidence Based Health Care Workbook: “An important challenge for medical educators… is to recognize that the competent student (and clinician!) is one who knows how to cope with an immense and rapidly changing body of knowledge and not one who excels in recalling the traditional or memorizing the ephemeral. The deans of medical and nursing schools must develop an infrastructure that allows problem-based, self-directed learning methods to develop within the didactic, lecture- based curricula, which have seen no fundamental changes for two centuries or more.”

268. ADVANCED EBM: ADVANCED APPLICATIONS APPLY METHODS TO… Care of the individual patient Team protocols Hospital or practice guidelines Continued Learning: problem-based approach Teaching

269. SELF-DIRECTED LEARNING JAMA Series of User Guides “Clinical Epidemiology: A Basic Science for Clinical Medicine” Week-long workshops On-the-job learning (in your own practice)

271. EVIDENCE BASED MEDICINE A new approach to clinical care and research Developed and presented by Judy Tarselli, RN Dubai, UAE Karachi, Pakistan October 2003 Organized by NKF cyberNephrology University of Alberta, Canada University of Alberta, Canadawww.cyberNephrology.org Special thanks to our sponsors at Janssen-Cilag of Dubai

272. EVIDENCE BASED MEDICINE A new approach to clinical care and research Developed and presented by Judy Tarselli, RN Dubai, UAE Karachi, Pakistan October 2003 Organized by NKF cyberNephrology University of Alberta, Canada University of Alberta, Canadawww.cyberNephrology.org Special thanks to our sponsors at Janssen-Cilag of Dubai