1. Evidence-Based Practice Introduction to methods and searching for Librarians Ann McKibbon MLS PhD McMaster University mckib@mcmaster.ca

2. Rules for the day Have fun Stretch your minds Make clinical decisions Ask questions and make comments Make mistakes and say dumb things Develop some new skills Stop me for breaks

3. Morning Agenda Introduction to EBM EBM and the question EBM and searching/retrieval –Categories of care –How are the similar –How are they different EBM and the article –Types of articles –What makes a “good’ one

4. Agenda How they differ? What makes them strong? How do I find them? –Therapy –Diagnosis –Prognosis –Etiology/harm –Systematic reviews

5. Current Working (Pragmatic) Definition …a set of tools, resources, and procedures …for finding current best evidence from research …and applying this evidence …for decision making with respect to the care of individual patients (EBM, EBN, EBD, EBHC) the care of groups of people (EBPublic Health) the running of your library (EBL) raising your child (EBParenting) …taking into account the situation, culture, resources, and common sense

6. Historical Roots of EBP Daniel—first clinical trial with Shadrack, Meshack, and Abednigo The results? At the end of 10 days, they "appeared better and fatter than all the young men who had been eating the royal rations" (v:15), and …the king found them in learning and wisdom to be "ten times better than all the magicians and enchanters in his whole kingdom" (vs:18-19).

7. Ibn Sina (981-1037) Rules of drug evaluation The drug must have a specific defined mode of action It must be tested on a well defined disease The time of action must be observed The effect of the drug must been seen to occur consistently in many cases The experimentation must be done on the human body (horses or lions may react differently than a human)

8. Historical Roots of EBP Lind—scurvy Late 1700s 12 sailors Dramatic results Slow implementation

9. Osler—Information It is astonishing with how little reading a doctor can practice medicine, but it is not astonishing how badly he may do it.

10. Historical Roots of EBP First randomized trial --not enough drug in UK --George Orwell not eligible for trial

11. EB Health Care—First Version Patient Evidence Clinical Expertise

12. EB Health Care—New Version Clinical Circumstances Clinical Experience Patient preferences Evidence and actions

13. 5 Steps of EBHC Framing the question appropriate to the needs of the patient Finding the evidence Evaluating the evidence (critical appraisal) Making and doing the decision Evaluation of the whole process

14. Critical appraisal Three-step process Find out how good (strong) the evidence is (assessment of the methods) Find out what the results are and how strong they are Figure out how our patient (or patient group) and settings matches with the study patients and setting

15. Critical appraisal Go to first page of supplemental package Pictorial representation of the EB process from a clinical perspective Most of the steps are done intuitively and not on paper

16. Why our searching skills are important

17. New Awareness of the Importance of Evidence and Difficulties to Find/Accumulate Systematic reviews and meta-analyses –Presence of search strategies a measure of quality –Expert searching—have we lost our edge? –Seeing a true collaboration (work as equals) –Searching and data management Cochrane and Campbell Collaborations, DARE –Librarians full members with own sections on methods CADTH, AHRQ and Technology Assessment –As above (Jessie McGowan) Guidelines and Care Maps –Librarians true partners (Ruth Holst) Professional Societies hire librarians/contract for services –CFPC, CMA, ACOG, AAN

18. Need for new information tools and techniques Books and book-like products –Clinical Evidence has much librarian input—searching etc –PIER from ACP Summary Journals –ACP Journal Club, EBM, EBN Web pages and sites –MEDLINEPLUS and Go Local implementations –Provide sites and production New products –OVID EBM Reviews and other aggregated services –PubMED Clinical Queries –BMJUpDates+ New services

19. New Respect for our Abilities / Collections Grants –Preparation, editing of grants –Teaching of grant writing skills (Wessel & Pitt modules) Institutional Review Boards –Membership –Standards (Kate Oliver) New service opportunities –Scherrer and publication process –Code Yellow—Library 911 Teaching –EBM workshops in Hamilton and Denver Limited only by our –Imagination –Ability to recognize and create opportunities

20. Clinical Research Move now to looking at clinical research… …the kind of research that is strong enough and applicable enough for use in making/changing clinical decisions

21. Clinical Research Question based—once question formed Methods –Observation –Manipulation/experimental Which is stronger to do? Which is easier to do? Do we need both? Why?

22. Observational or Experimental? What is the process that elderly people go through as they come to terms with living in an assisted living arrangement? Does yoga improve fatigue levels in people recovering from cancer? How effective is this appetite reducing drug in relation to exercise? Do suduku puzzles protect me from developing Alzheimer disease? What is my life expectancy now that you have told me I have ALS? Is this blood test as effective as stomach biopsy at telling me that I have celiac disease?

23. Publishing Wedge (therapy) Idea Idea development Laboratory Animal Early humanPhase I Middle human Phase II Late humanPhase III

24. Categories of clinical literature Original studies Therapy Diagnosis Prognosis Etiology Clinical prediction guides Differential diagnosis Qualitative studies Syntheses Systematic reviews/ meta-analyses Clinical practice guidelines Economic studies Decision analyses

25. Similarities across categories Done to answer legitimate and important problems and issues Meet standards –Ethical –Moral –Record keeping and reporting –Publishing standards and procedures

26. Similarities across categories Question based Preplanned Comparative Patients/participants Results and statistics Settings Cultures/health care systems Conflict of interest/disclosure statements Funding sources

27. Clinical Question For young children, is a smoke detector/fire alarm that uses a recording of his or her mother’s voice more effective at awakening the child and shortening time to evacuation as compared to a standard tone-based smoke detector?

28. Settings Important for assessment of match between patient and article Most often primary care, tertiary care, hospital, office (UK surgery), nursing home, university medical center, chronic care facilities, home care Compare the “typical” headache seen in above settings

29. Cultures/health care systems For profit Not for profit Managed care Health maintenance organizations Medicare/Medicaid Veterans Affairs Hospitals Socialized medicine Two tier vs. three tier

30. Health care systems—assignment What health care system does each country have? What “type” of research comes from each country? Canada Israel Sweden United Kingdom Australia The Netherlands United States

31. Funding Sources Complex and costly issue Competing demands with “for profit” funders vs. “not for profit” funders Kjaergard and Als-Nilesen showed that – “in pharmacological and nonpharmacological randomised controlled trials from 12 specialties financial competing interests were significantly associated with authors’ conclusions” – “personal, academic and political competing interests were not significantly associated with authors’ conclusions”.

32. Conflict of Interest Researchers or authors of the study or its report stand to gain (probably financially) from certain results of the study. Watch for this. Should be included in the article—usually at the end and in a small font.

33. Differences All articles same for a few features –Question based –Preplanned –Comparative –Patients/participants –Settings –Cultures/health care systems –Funding sources Then different….

34. Therapy

35. Therapy/Treatment More of these types of studies than any other Methodology is pretty well established for conducting trials and presenting results Indexing (and retrieval) excellent

36. Alternate Names Therapy/Treatment (medicine) Intervention (nursing) Prevention and control Prevention –Primary prevention –Secondary prevention –Tertiary prevention Quality improvement Management???

37. How to do a Therapy Trial 2 or more groups Each group gets 1 intervention All groups followed over time At the end of the trial groups compared to assess outcomes

38. Things to Look for in a Therapy Trial Common sense Allocation concealment Random allocation Blinding Follow-up Sensible and important outcomes

39. Allocation Concealment Done before and during randomization No one with any influence on who goes into which group can have any knowledge of which group is next Differences among outcomes if this is not done—more favourable outcomes

40. Random Allocation You can randomize: parts of people (e.g., arms, warts) whole people families hospitals or wards (cluster) towns

41. Methods for Random Allocation Best Computer methods that do not allow for manipulation of randomization An agency that has no involvement in patient recruitment such as a pharmacy department An external trials office that entails calling into a central registration office and providing details of patient before the randomization is done Sealed, opaque, consecutively numbered envelopes with external checking

42. Blinding Individuals involved in a study (e.g., patients, investigators, research staff) do not know who is assigned to treatment or control groups. Why: Individual expectations can influence study outcomes

43. Types of Blinding Single, double, triple blinding although many more could be listed Masked, dummy also used –Patients –Care providers –Study personnel Data collectors Outcome assessors –Data analysts –Report writers –Sponsors

44. Placebo To aid in blinding some trial participants may get something that is not really a treatment or for “real” –Sugar pill in vitamin C trials –Sham ultrasound in treating BPH and both real and sham kept “equivalent” by heating pads placed over treatment sites –Can be for assessors in addition to patient as in fake blood for patients in a scope vs full surgery trial

45. Follow-up Concerns the number of participants who completed the study Look for withdrawals, drop outs, or those who were lost 80% is magic number Common sense

46. Clinical Question Pain is a complex challenge at the best of times and is especially difficult for children. A hospital committee has been formed that wants to look at alternatives to simply increasing the doses of pain meds. Someone has suggested that video games may distract kids with substantial burns from thinking about their pain. Does the following article support this assertion?

47. MeSH ResearchComparative study Clinical protocolsPlacebos* Feasibility studiesClinical trials as topic Pilot projects Clinical trials as topic, I-IV Research design* Multicenter studies Double-blind method* Randomized controlled trials Meta-analysis as topic Patient selectionTreatment outcome Random allocation* Single-blind method Sample size

48. Publication Types Clinical trial Clinical trial, phase I Clinical trial, phase II Clinical trial, phase III Clinical trial, phase IV Randomized controlled trial Controlled clinical trial Multicenter study Meta-analysis

49. Subheadings Therapy (explodable) Surgery Radiotherapy Diet therapy Psychology Therapeutic use (explodable) Administration and dosage

50. Textwords Random:Double blind: Double-dummyMask: Sham:Placebo: Control: trial:Efficacy Effectiveness

51. Diagnosis and Screening

52. Disease/condition Present? Diagnosis This patient has signs and symptoms that suggest diabetes. Does she have it? Signs and symptoms are present that warrant action. Screening We are going to check all students in this school to see if the head lice has spread from room 2. No symptoms are present but because of the population we are going to assess all of them.

53. Diagnostic Decision Three choices after assessing patient: Do nothing for a while—I am not sure at all if the patient has a condition—low probability of a positive diagnosis Treat right away—I am sure beyond any doubt that the patient has the condition—the diagnostic tests will give me no more information Start doing diagnostic tests

54. Diagnostic Decision Probability of disease 0% testing threshold treatment threshold 100% Wait TestTreat

55. Diagnostic Decision Wait TestTreat Probability of disease 0% testing threshold  treatment threshold 100% Positive test results moves     Negative test results moves    

56. Diagnosis study—example Problem: Is this incontinence urge or stress? One treated with drugs, other behaviour Possible solution: questionnaire vs urologist? Tested 301 women some incontinence. Test results after every women got both 75% of time positive when have incontinence 77% of time negative when no incontinence

57. Diagnosis/Screening Does this person have or not have a specific disease or condition? Can questionnaires in family medicine settings screen for: –eating disorders –Depression –domestic violence –Alzheimer disease –drinking problems

58. Old Test vs New Test Need patients to have spectrum of disease (none to severe) Everyone gets both tests Old test is often invasive, time consuming, costly, or has risks involved Can only do if “gold” test available or can be “rigged”

59. Things to look for in a diagnosis study Old test vs new test Blinding of assessment of results of both tests

60. Blinding of Test Reading Absolutely crucial for evaluation of diagnostic tests Exceptions are things like laboratory tests that do not involve personal biases Blinding is almost NEVER indexed by NLM nor is it included in the abstract of the articles. Often hard to find in the body of the article (methods section)

61. A good test…. …is positive when it should be positive AND negative when it should be negative... Sensitivity and specificity Positive and negative predictive values False positive and negative reactions Positive and negative likelihood ratio

62. Calculating diagnosis numbers Disease is present Disease is absent Test shows positive ( disease may be present ) A B Test shows negative ( probably no disease ) C D A + CB + D

63. Sensitivity and Specificity Sensitivity--- ---test is positive when it should be positive Specificity--- ---test is negative when it should be negative

64. Predictive Values Positive--- ---proportion of people with positive results who actually have the disease Negative--- ---proportion of people with negative results who do NOT have the disease

65. False Positive/Negative Results False positive--- ---test is positive when the person does not have the disease (labeling) False negative--- ---test is negative when the person does have the disease (lose time)

66. Likelihood Ratios The likelihood ratio for a positive result (LR+) tells you how likely you are to have a positive test if you HAVE the disease The likelihood ratio for a negative result (LR-) tells you how likely you are to have a negative test if you do NOT have the disease.

67. Diagnostic Decision Wait TestTreat Probability of disease testing threshold  treatment threshold 0%100% Positive test results moves     Negative test results moves     Likelihood ratios are the SLIDERS on this scale!

68. Concepts Accuracy Bayes theorem Diagnosis False negative False positive Negative predictive value Positive predictive value ROC curves Likelihood ratios Sensitivity Specificity Nomogram Screening

69. MeSH Sensitivity a#d specificity* Predictive values of tests ROC curves Diagnostic errors False negative reactions False positive reactions Observer variations Likelihood functions Diagnosis, differential* Reproducibility of results

70. Publication Types

71. Subheadings Diagnosis (explodable) Diagnostic use (explodable)

72. Textwords Sensitivit: Specificit: Predictive value: False positive False negative Likelihood ratio Accura:

73. Prognosis

74. Prognosis/Natural History What is going to happen to me over the next period of time now that I have been diagnosed with… Patient preference for this type of information and not therapy. Should I treat (or choose to be treated) rather than what is best to treat with. (Scoliosis and prostate cancer)

75. Prognosis—Example Do we know the natural history or prognosis for patients who have have been diagnosed with Parkinson disease? 297 patients (181 men) with 1731 visits over 6.4 years (mean) showed a variable course of the disease and its symptoms

76. Prognosis vs Natural History Traditional difference… Natural history is what happens to untreated disease over time…. Prognosis is what happens to treated disease…

77. Prognosis vs Natural History Natural History Biological onset at cellular level Early diagnosis possible (screening) Usual diagnosis Outcomes Prognosis Usual diagnosis Outcomes

78. Prognosis Methodology Inception cohort study Group of persons assembled early (or at least at a uniform point) in the timing of their disease and followed over time.

79. Prognosis Methodology Inception cohort study… Follow up 80% or better The follow-up time should be consistent with the demands of the disease Common sense rules

80. Prognosis Numbers Raw rates of disease progression (14% had a subsequent myocardial infarction within 5 years of follow up) Relative risks Hazards ratios Odds ratio Standardized mortality ratios

81. Prognostic vs Risk Factors Prognostic factors Some aspect that an individual has that can modify how that disease will play out—e.g., age of person with CAD, breast cancer stage Risk factors Some aspect that an individual has that may affect whether that person is more or less prone to acquire a disease or condition, e.g., family history of CAD, Al and Alzheimer’s

82. Prognosis Terms AdjustmentNatural History Cohort studyPrevalence InceptionPrognosis IncidencePrognostic factor Longitudinal studies Prospective studies MorbidityRisk factors Mortality rates, ratios

83. MeSH Cohort studies*Survival XXXXXX longitudinal studies prospective studies follow-up studies Prognosis* Morbidity incidence prevalence Mortality Survival analysis Disease progression* Time factors* Age factors* Sex factors*

84. Publication Types

85. Subheadings Mortality Epidemiology (sometimes)

86. Textwords Natural historyPrognos: Inception cohortClinical course Predict:Predictive value OutcomePrognostic factor Course

87. Alternative Source For short-term prognosis information, randomized controlled trials often have information that can be used when desperate. This is especially true for placebo-controlled or usual-care arms of studies.

88. Etiology/Causation/Harm

89. Etiology or causation is the study of what causes, or what increases or decreases the risk for a disease or condition. This can be either positive (protective) or negative (harmful). e.g., social support at work is associated with fewer short-term psychiatric absences giving up driving is associated with depressive symptoms in older adults.

90. Etiology/Causation/Harm Generally looking at Exposures to causative/protective agents Outcomes Time (can be now, in the future, or looking back) Other factors that can affect outcomes Need a lot of creativity and common sense Many long-term etiology studies come from countries with socialized medicine.

91. Q: Does the use of cell phones while driving cause accidents? Exposures? Outcomes? What groups?

92. Q: Does the use of cell phones while driving cause accidents? Randomized controlled trial large group of persons who are told whether they will use cell phone during driving for long time…..

93. Q: Does the use of cell phones while driving cause accidents? Cohort study a group of persons who uses phones while driving is compared with another group who do not use phones while driving…

94. Q: Does the use of cell phones while driving cause accidents? Case-control study people who have had automobile accidents are compared with people who have not had accidents and both groups are studied to see who used phones during driving…

95. Q: Does the use of cell phones while driving cause accidents? Statistically adjusted groups (cross sectional study) Data on some persons with automobile accidents were collected and compared with data from some persons with no accidents and both groups have cell phone use rates determined….

96. Etiology study types TypeTimeQualityNumber RCTfuture****** Cohortnow****** Case-controlpast****** Statisticallypast-infinity adjusted groups (cross sectional studies)

97. Etiology Issues Association is not necessarily causation Ethics Logistics Blinding 2 C’s—Common sense and Creativity

98. Association vs Causation Just because two things occur at the same time does not mean that they are causal-- Higher intake of ice cream and higher rates of drowning occur in the summer—linked? Don’t think so…... Poor quality health and low socioeconomic status……

99. Ethics Sometimes you cannot allocate persons to exposures –Smoking –Divorce –Genetic disposition –High socioeconomic status –Can do this however for drug adverse effects, social programs

100. Logistics Groups need to be as similar as possible to account for confounding For power line studies….. Smoking and drinking issues… Another set of “intertwined” issues? Need for creative problem solving

101. Blinding Blinding is crucial especially for the case- control studies. Blinding must be 2-fold here: –forming groups without knowing exposures (cell phone use in cars) –assessing exposure without knowing disease/exposure status (automobile accidents)

102. Relative Risk (RR) Used for RCTs and cohort studies (prospective) Comparison of rates of developing the disease/ condition in the 2 groups of people with and without the risk factor (We know exposures) Weight gain and coronary heart disease in women (400 women in our sample) if gain > 15 lb106/200 if gain < 15 lb 56/200 RR= (106/200)/(56/200) = 1.9

103. Concepts Association Blinding Causation Case-control study Cohort study Confounders Cross sectional studies Odds ratio Prospective studies Relative risk Retrospective studies Risk Risk factors

104. MeSH Case-control studies* Retrospective studies Cohort studies* Longitudinal studies Prospective studies Follow-up studies Cross sectional studies Risk* Risk assessment Risk factors Odds ratio

105. Publication Types

106. Subheadings Etiology (explodable) Epidemiology (for distributions, causes, and attributes of disease)

107. Textwords Cohort Case control Risk Odds ratio Causation or causal: Relative risk Etiol: or Aetiol:

108. Systematic Reviews and Meta-Analyses Two types of review articles exist: Narrative reviews Systematic reviews –Systematic reviews –Meta-analyses of study data –Meta-analyses using individual patient data

109. Systematic Reviews Medicine did not “invent” or develop Fully developed by psychology, education, and related disciplines Early 1900s –Pearson and enteric fever in the British Army –NIH (Hygiene Laboratory) report People: –Eugene Glass was one of the first developers –Tom Chalmers was one of the first medical developers (On Golden Pond) –Archie Cochrane plus Tom and Iain Chalmers

110. Systematic vs Narrative Team approach Narrow specific purpose Methods drive the process Inclusion/exclusion criteria Clinically useful One major author Broad purpose may not always be stated No methods on how articles picked No inclusion/ exclusion criteria Ecuationally useful

111. Systematic Reviews Must have Purpose why done Search strategy in detail Inclusion and exclusion criteria for study selection

112. Why do one? Too much data Too little data Resolve discrepancies “tighten up” estimates of effects of treatments, exposures, etc Analyze patient subgroups Plan for new studies—similar or “next generation” studies Provide data for certain types of studies such as economic studies, decision analyses, or clinical practice guidelines

113. Steps Problem formulation Identify and select articles (searching and retrieval) Data extraction Analysis and decision if meta-analysis is appropriate (clinical and statistical test) Presentation of results

114. Statistical Concepts Homogeniety Heterogeniety Weighting Pooling Effect sizes

115. Statistical Concepts Same as for other study types but often qualified with “combining” phrases combined odds ratio pooledrelative risk weighted hazards ratio typicalmortality rate summary estimatessensitivity

116. MEDLINE Difference Simplistic difference (and hard to differentiate between the two) Review—does a summary of existing knowledge Meta-analysis—produces new knowledge

117. Meta-analysis MeSH A quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc., with application chiefly in the areas of research and medicine. Clinical trials overview, data pooling,

118. Meta-analysis Previous Indexing Follow up studies Outcome and process assessment Research Research design Statistics

119. Meta-analysis Publication Type Works consisting of studies using a quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc. It is often an overview of clinical trials. It is usually called a meta- analysis by the author or sponsoring body and should be differentiated from reviews of literature.

120. Review Literature MeSH Published materials which provide an examination of recent or current literature. Review articles can cover a wide range of subject matter at various levels of completeness and comprehensiveness based on analyses of literature that may include research findings. The review may reflect the state of the art. It also includes reviews as a literary form.

121. Systematic Reviews Terms Effect sizes Heterogeniety Homogeniety Meta-analysis of study data Meta-analysis of individual pt data Pooling Narrative review Summary estimates Systematic reviews Typical Weighting

122. MeSH Randomized controlled trials Clinical trials Controlled trials Meta-analysis

123. Publication Types Meta-analysis Review iff ANDed with MeSH or textwords

124. Subheadings

125. Textwords Meta-analy: Metanal: Metaanal: Systematic review or overview Overview (careful) Quantitative review or overview Methodologic: review or overview Heterogeniety Homogeniety Medline Psychinfo Psycinfo Embase

126. Food for Thought Sometimes a rare condition that is being written up as a case report will include a substantial amount of background material. Don’t necessarily cross out a case-report if you are desperate for review type material.

127. Alternative Sources PubMed Clinical Queries—Review articles Cochrane—note only for RCTS –Cochrane reviews –Dare –Clinical Trials DARE (Database of Reviews of Effectiveness— librarian run project) Campbell Collaboration TRIP database—one stop shopping

128. Systematic Reviews Quiz 1 Systematic reviews are the same as ordinary reviews only bigger? Based on: Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001;322:98-101.

129. Systematic Reviews Quiz 2 Systematic reviews contain only randomized controlled trials.

130. Systematic Reviews Quiz 3Only medical topics can be studied in systematic reviews. –More money on schools improved educational outcomes? –Do men or women make better leaders? –Does the sexual orientation of the parent matter to the child’s well being? –Are fathers more likely to treat their children differently than mothers? –Is job absenteeism an indicator of job dissatisfaction? –Are jurors influenced by the race of the defendants? –Is there a relation between poverty and violence?

131. Systematic Reviews Quiz 4Systematic reviews can be done well without experienced information personnel or good library support.

132. Systematic Reviews Quiz 5Systematic reviews have no relevance to the real world

133. Systematic Reviews Quiz 6Systematic review necessarily involve statistical syntheses.

134. Systematic Reviews Quiz 7Systematic reviews have to be done by experts with a lot of training in methodologies.

135. Systematic Reviews Quiz 8Systematic reviews are a substitute for doing good quality individual studies.

136. Systematic Reviews Quiz 9You can be part of a team doing a systematic review.