Presentation on theme: "Evidence-Based Practice Introduction to methods and searching for Librarians Ann McKibbon MLS PhD McMaster University"— Presentation transcript:
Evidence-Based Practice Introduction to methods and searching for Librarians Ann McKibbon MLS PhD McMaster University firstname.lastname@example.org
Rules for the day Have fun Stretch your minds Make clinical decisions Ask questions and make comments Make mistakes and say dumb things Develop some new skills Stop me for breaks
Morning Agenda Introduction to EBM EBM and the question EBM and searching/retrieval –Categories of care –How are the similar –How are they different EBM and the article –Types of articles –What makes a “good’ one
Agenda How they differ? What makes them strong? How do I find them? –Therapy –Diagnosis –Prognosis –Etiology/harm –Systematic reviews
Current Working (Pragmatic) Definition …a set of tools, resources, and procedures …for finding current best evidence from research …and applying this evidence …for decision making with respect to the care of individual patients (EBM, EBN, EBD, EBHC) the care of groups of people (EBPublic Health) the running of your library (EBL) raising your child (EBParenting) …taking into account the situation, culture, resources, and common sense
Historical Roots of EBP Daniel—first clinical trial with Shadrack, Meshack, and Abednigo The results? At the end of 10 days, they "appeared better and fatter than all the young men who had been eating the royal rations" (v:15), and …the king found them in learning and wisdom to be "ten times better than all the magicians and enchanters in his whole kingdom" (vs:18-19).
Ibn Sina (981-1037) Rules of drug evaluation The drug must have a specific defined mode of action It must be tested on a well defined disease The time of action must be observed The effect of the drug must been seen to occur consistently in many cases The experimentation must be done on the human body (horses or lions may react differently than a human)
Historical Roots of EBP Lind—scurvy Late 1700s 12 sailors Dramatic results Slow implementation
Osler—Information It is astonishing with how little reading a doctor can practice medicine, but it is not astonishing how badly he may do it.
Historical Roots of EBP First randomized trial --not enough drug in UK --George Orwell not eligible for trial
EB Health Care—First Version Patient Evidence Clinical Expertise
EB Health Care—New Version Clinical Circumstances Clinical Experience Patient preferences Evidence and actions
5 Steps of EBHC Framing the question appropriate to the needs of the patient Finding the evidence Evaluating the evidence (critical appraisal) Making and doing the decision Evaluation of the whole process
Critical appraisal Three-step process Find out how good (strong) the evidence is (assessment of the methods) Find out what the results are and how strong they are Figure out how our patient (or patient group) and settings matches with the study patients and setting
Critical appraisal Go to first page of supplemental package Pictorial representation of the EB process from a clinical perspective Most of the steps are done intuitively and not on paper
New Awareness of the Importance of Evidence and Difficulties to Find/Accumulate Systematic reviews and meta-analyses –Presence of search strategies a measure of quality –Expert searching—have we lost our edge? –Seeing a true collaboration (work as equals) –Searching and data management Cochrane and Campbell Collaborations, DARE –Librarians full members with own sections on methods CADTH, AHRQ and Technology Assessment –As above (Jessie McGowan) Guidelines and Care Maps –Librarians true partners (Ruth Holst) Professional Societies hire librarians/contract for services –CFPC, CMA, ACOG, AAN
Need for new information tools and techniques Books and book-like products –Clinical Evidence has much librarian input—searching etc –PIER from ACP Summary Journals –ACP Journal Club, EBM, EBN Web pages and sites –MEDLINEPLUS and Go Local implementations –Provide sites and production New products –OVID EBM Reviews and other aggregated services –PubMED Clinical Queries –BMJUpDates+ New services
New Respect for our Abilities / Collections Grants –Preparation, editing of grants –Teaching of grant writing skills (Wessel & Pitt modules) Institutional Review Boards –Membership –Standards (Kate Oliver) New service opportunities –Scherrer and publication process –Code Yellow—Library 911 Teaching –EBM workshops in Hamilton and Denver Limited only by our –Imagination –Ability to recognize and create opportunities
Clinical Research Move now to looking at clinical research… …the kind of research that is strong enough and applicable enough for use in making/changing clinical decisions
Clinical Research Question based—once question formed Methods –Observation –Manipulation/experimental Which is stronger to do? Which is easier to do? Do we need both? Why?
Observational or Experimental? What is the process that elderly people go through as they come to terms with living in an assisted living arrangement? Does yoga improve fatigue levels in people recovering from cancer? How effective is this appetite reducing drug in relation to exercise? Do suduku puzzles protect me from developing Alzheimer disease? What is my life expectancy now that you have told me I have ALS? Is this blood test as effective as stomach biopsy at telling me that I have celiac disease?
Publishing Wedge (therapy) Idea Idea development Laboratory Animal Early humanPhase I Middle human Phase II Late humanPhase III
Categories of clinical literature Original studies Therapy Diagnosis Prognosis Etiology Clinical prediction guides Differential diagnosis Qualitative studies Syntheses Systematic reviews/ meta-analyses Clinical practice guidelines Economic studies Decision analyses
Similarities across categories Done to answer legitimate and important problems and issues Meet standards –Ethical –Moral –Record keeping and reporting –Publishing standards and procedures
Similarities across categories Question based Preplanned Comparative Patients/participants Results and statistics Settings Cultures/health care systems Conflict of interest/disclosure statements Funding sources
Clinical Question For young children, is a smoke detector/fire alarm that uses a recording of his or her mother’s voice more effective at awakening the child and shortening time to evacuation as compared to a standard tone-based smoke detector?
Settings Important for assessment of match between patient and article Most often primary care, tertiary care, hospital, office (UK surgery), nursing home, university medical center, chronic care facilities, home care Compare the “typical” headache seen in above settings
Cultures/health care systems For profit Not for profit Managed care Health maintenance organizations Medicare/Medicaid Veterans Affairs Hospitals Socialized medicine Two tier vs. three tier
Health care systems—assignment What health care system does each country have? What “type” of research comes from each country? Canada Israel Sweden United Kingdom Australia The Netherlands United States
Funding Sources Complex and costly issue Competing demands with “for profit” funders vs. “not for profit” funders Kjaergard and Als-Nilesen showed that – “in pharmacological and nonpharmacological randomised controlled trials from 12 specialties financial competing interests were significantly associated with authors’ conclusions” – “personal, academic and political competing interests were not significantly associated with authors’ conclusions”.
Conflict of Interest Researchers or authors of the study or its report stand to gain (probably financially) from certain results of the study. Watch for this. Should be included in the article—usually at the end and in a small font.
Differences All articles same for a few features –Question based –Preplanned –Comparative –Patients/participants –Settings –Cultures/health care systems –Funding sources Then different….
Therapy/Treatment More of these types of studies than any other Methodology is pretty well established for conducting trials and presenting results Indexing (and retrieval) excellent
Alternate Names Therapy/Treatment (medicine) Intervention (nursing) Prevention and control Prevention –Primary prevention –Secondary prevention –Tertiary prevention Quality improvement Management???
How to do a Therapy Trial 2 or more groups Each group gets 1 intervention All groups followed over time At the end of the trial groups compared to assess outcomes
Things to Look for in a Therapy Trial Common sense Allocation concealment Random allocation Blinding Follow-up Sensible and important outcomes
Allocation Concealment Done before and during randomization No one with any influence on who goes into which group can have any knowledge of which group is next Differences among outcomes if this is not done—more favourable outcomes
Random Allocation You can randomize: parts of people (e.g., arms, warts) whole people families hospitals or wards (cluster) towns
Methods for Random Allocation Best Computer methods that do not allow for manipulation of randomization An agency that has no involvement in patient recruitment such as a pharmacy department An external trials office that entails calling into a central registration office and providing details of patient before the randomization is done Sealed, opaque, consecutively numbered envelopes with external checking
Blinding Individuals involved in a study (e.g., patients, investigators, research staff) do not know who is assigned to treatment or control groups. Why: Individual expectations can influence study outcomes
Types of Blinding Single, double, triple blinding although many more could be listed Masked, dummy also used –Patients –Care providers –Study personnel Data collectors Outcome assessors –Data analysts –Report writers –Sponsors
Placebo To aid in blinding some trial participants may get something that is not really a treatment or for “real” –Sugar pill in vitamin C trials –Sham ultrasound in treating BPH and both real and sham kept “equivalent” by heating pads placed over treatment sites –Can be for assessors in addition to patient as in fake blood for patients in a scope vs full surgery trial
Follow-up Concerns the number of participants who completed the study Look for withdrawals, drop outs, or those who were lost 80% is magic number Common sense
Clinical Question Pain is a complex challenge at the best of times and is especially difficult for children. A hospital committee has been formed that wants to look at alternatives to simply increasing the doses of pain meds. Someone has suggested that video games may distract kids with substantial burns from thinking about their pain. Does the following article support this assertion?
MeSH ResearchComparative study Clinical protocolsPlacebos* Feasibility studiesClinical trials as topic Pilot projects Clinical trials as topic, I-IV Research design* Multicenter studies Double-blind method* Randomized controlled trials Meta-analysis as topic Patient selectionTreatment outcome Random allocation* Single-blind method Sample size
Publication Types Clinical trial Clinical trial, phase I Clinical trial, phase II Clinical trial, phase III Clinical trial, phase IV Randomized controlled trial Controlled clinical trial Multicenter study Meta-analysis
Subheadings Therapy (explodable) Surgery Radiotherapy Diet therapy Psychology Therapeutic use (explodable) Administration and dosage
Disease/condition Present? Diagnosis This patient has signs and symptoms that suggest diabetes. Does she have it? Signs and symptoms are present that warrant action. Screening We are going to check all students in this school to see if the head lice has spread from room 2. No symptoms are present but because of the population we are going to assess all of them.
Diagnostic Decision Three choices after assessing patient: Do nothing for a while—I am not sure at all if the patient has a condition—low probability of a positive diagnosis Treat right away—I am sure beyond any doubt that the patient has the condition—the diagnostic tests will give me no more information Start doing diagnostic tests
Diagnostic Decision Probability of disease 0% testing threshold treatment threshold 100% Wait TestTreat
Diagnostic Decision Wait TestTreat Probability of disease 0% testing threshold treatment threshold 100% Positive test results moves Negative test results moves
Diagnosis study—example Problem: Is this incontinence urge or stress? One treated with drugs, other behaviour Possible solution: questionnaire vs urologist? Tested 301 women some incontinence. Test results after every women got both 75% of time positive when have incontinence 77% of time negative when no incontinence
Diagnosis/Screening Does this person have or not have a specific disease or condition? Can questionnaires in family medicine settings screen for: –eating disorders –Depression –domestic violence –Alzheimer disease –drinking problems
Old Test vs New Test Need patients to have spectrum of disease (none to severe) Everyone gets both tests Old test is often invasive, time consuming, costly, or has risks involved Can only do if “gold” test available or can be “rigged”
Things to look for in a diagnosis study Old test vs new test Blinding of assessment of results of both tests
Blinding of Test Reading Absolutely crucial for evaluation of diagnostic tests Exceptions are things like laboratory tests that do not involve personal biases Blinding is almost NEVER indexed by NLM nor is it included in the abstract of the articles. Often hard to find in the body of the article (methods section)
A good test…. …is positive when it should be positive AND negative when it should be negative... Sensitivity and specificity Positive and negative predictive values False positive and negative reactions Positive and negative likelihood ratio
Calculating diagnosis numbers Disease is present Disease is absent Test shows positive ( disease may be present ) A B Test shows negative ( probably no disease ) C D A + CB + D
Sensitivity and Specificity Sensitivity--- ---test is positive when it should be positive Specificity--- ---test is negative when it should be negative
Predictive Values Positive--- ---proportion of people with positive results who actually have the disease Negative--- ---proportion of people with negative results who do NOT have the disease
False Positive/Negative Results False positive--- ---test is positive when the person does not have the disease (labeling) False negative--- ---test is negative when the person does have the disease (lose time)
Likelihood Ratios The likelihood ratio for a positive result (LR+) tells you how likely you are to have a positive test if you HAVE the disease The likelihood ratio for a negative result (LR-) tells you how likely you are to have a negative test if you do NOT have the disease.
Diagnostic Decision Wait TestTreat Probability of disease testing threshold treatment threshold 0%100% Positive test results moves Negative test results moves Likelihood ratios are the SLIDERS on this scale!
Prognosis/Natural History What is going to happen to me over the next period of time now that I have been diagnosed with… Patient preference for this type of information and not therapy. Should I treat (or choose to be treated) rather than what is best to treat with. (Scoliosis and prostate cancer)
Prognosis—Example Do we know the natural history or prognosis for patients who have have been diagnosed with Parkinson disease? 297 patients (181 men) with 1731 visits over 6.4 years (mean) showed a variable course of the disease and its symptoms
Prognosis vs Natural History Traditional difference… Natural history is what happens to untreated disease over time…. Prognosis is what happens to treated disease…
Prognosis vs Natural History Natural History Biological onset at cellular level Early diagnosis possible (screening) Usual diagnosis Outcomes Prognosis Usual diagnosis Outcomes
Prognosis Methodology Inception cohort study Group of persons assembled early (or at least at a uniform point) in the timing of their disease and followed over time.
Prognosis Methodology Inception cohort study… Follow up 80% or better The follow-up time should be consistent with the demands of the disease Common sense rules
Prognosis Numbers Raw rates of disease progression (14% had a subsequent myocardial infarction within 5 years of follow up) Relative risks Hazards ratios Odds ratio Standardized mortality ratios
Prognostic vs Risk Factors Prognostic factors Some aspect that an individual has that can modify how that disease will play out—e.g., age of person with CAD, breast cancer stage Risk factors Some aspect that an individual has that may affect whether that person is more or less prone to acquire a disease or condition, e.g., family history of CAD, Al and Alzheimer’s
Alternative Source For short-term prognosis information, randomized controlled trials often have information that can be used when desperate. This is especially true for placebo-controlled or usual-care arms of studies.
Etiology or causation is the study of what causes, or what increases or decreases the risk for a disease or condition. This can be either positive (protective) or negative (harmful). e.g., social support at work is associated with fewer short-term psychiatric absences giving up driving is associated with depressive symptoms in older adults.
Etiology/Causation/Harm Generally looking at Exposures to causative/protective agents Outcomes Time (can be now, in the future, or looking back) Other factors that can affect outcomes Need a lot of creativity and common sense Many long-term etiology studies come from countries with socialized medicine.
Q: Does the use of cell phones while driving cause accidents? Exposures? Outcomes? What groups?
Q: Does the use of cell phones while driving cause accidents? Randomized controlled trial large group of persons who are told whether they will use cell phone during driving for long time…..
Q: Does the use of cell phones while driving cause accidents? Cohort study a group of persons who uses phones while driving is compared with another group who do not use phones while driving…
Q: Does the use of cell phones while driving cause accidents? Case-control study people who have had automobile accidents are compared with people who have not had accidents and both groups are studied to see who used phones during driving…
Q: Does the use of cell phones while driving cause accidents? Statistically adjusted groups (cross sectional study) Data on some persons with automobile accidents were collected and compared with data from some persons with no accidents and both groups have cell phone use rates determined….
Etiology study types TypeTimeQualityNumber RCTfuture****** Cohortnow****** Case-controlpast****** Statisticallypast-infinity adjusted groups (cross sectional studies)
Etiology Issues Association is not necessarily causation Ethics Logistics Blinding 2 C’s—Common sense and Creativity
Association vs Causation Just because two things occur at the same time does not mean that they are causal-- Higher intake of ice cream and higher rates of drowning occur in the summer—linked? Don’t think so…... Poor quality health and low socioeconomic status……
Ethics Sometimes you cannot allocate persons to exposures –Smoking –Divorce –Genetic disposition –High socioeconomic status –Can do this however for drug adverse effects, social programs
Logistics Groups need to be as similar as possible to account for confounding For power line studies….. Smoking and drinking issues… Another set of “intertwined” issues? Need for creative problem solving
Blinding Blinding is crucial especially for the case- control studies. Blinding must be 2-fold here: –forming groups without knowing exposures (cell phone use in cars) –assessing exposure without knowing disease/exposure status (automobile accidents)
Relative Risk (RR) Used for RCTs and cohort studies (prospective) Comparison of rates of developing the disease/ condition in the 2 groups of people with and without the risk factor (We know exposures) Weight gain and coronary heart disease in women (400 women in our sample) if gain > 15 lb106/200 if gain < 15 lb 56/200 RR= (106/200)/(56/200) = 1.9
Concepts Association Blinding Causation Case-control study Cohort study Confounders Cross sectional studies Odds ratio Prospective studies Relative risk Retrospective studies Risk Risk factors
Subheadings Etiology (explodable) Epidemiology (for distributions, causes, and attributes of disease)
Textwords Cohort Case control Risk Odds ratio Causation or causal: Relative risk Etiol: or Aetiol:
Systematic Reviews and Meta-Analyses Two types of review articles exist: Narrative reviews Systematic reviews –Systematic reviews –Meta-analyses of study data –Meta-analyses using individual patient data
Systematic Reviews Medicine did not “invent” or develop Fully developed by psychology, education, and related disciplines Early 1900s –Pearson and enteric fever in the British Army –NIH (Hygiene Laboratory) report People: –Eugene Glass was one of the first developers –Tom Chalmers was one of the first medical developers (On Golden Pond) –Archie Cochrane plus Tom and Iain Chalmers
Systematic vs Narrative Team approach Narrow specific purpose Methods drive the process Inclusion/exclusion criteria Clinically useful One major author Broad purpose may not always be stated No methods on how articles picked No inclusion/ exclusion criteria Ecuationally useful
Systematic Reviews Must have Purpose why done Search strategy in detail Inclusion and exclusion criteria for study selection
Why do one? Too much data Too little data Resolve discrepancies “tighten up” estimates of effects of treatments, exposures, etc Analyze patient subgroups Plan for new studies—similar or “next generation” studies Provide data for certain types of studies such as economic studies, decision analyses, or clinical practice guidelines
Steps Problem formulation Identify and select articles (searching and retrieval) Data extraction Analysis and decision if meta-analysis is appropriate (clinical and statistical test) Presentation of results
Statistical Concepts Same as for other study types but often qualified with “combining” phrases combined odds ratio pooledrelative risk weighted hazards ratio typicalmortality rate summary estimatessensitivity
MEDLINE Difference Simplistic difference (and hard to differentiate between the two) Review—does a summary of existing knowledge Meta-analysis—produces new knowledge
Meta-analysis MeSH A quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc., with application chiefly in the areas of research and medicine. Clinical trials overview, data pooling,
Meta-analysis Previous Indexing Follow up studies Outcome and process assessment Research Research design Statistics
Meta-analysis Publication Type Works consisting of studies using a quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc. It is often an overview of clinical trials. It is usually called a meta- analysis by the author or sponsoring body and should be differentiated from reviews of literature.
Review Literature MeSH Published materials which provide an examination of recent or current literature. Review articles can cover a wide range of subject matter at various levels of completeness and comprehensiveness based on analyses of literature that may include research findings. The review may reflect the state of the art. It also includes reviews as a literary form.
Systematic Reviews Terms Effect sizes Heterogeniety Homogeniety Meta-analysis of study data Meta-analysis of individual pt data Pooling Narrative review Summary estimates Systematic reviews Typical Weighting
Textwords Meta-analy: Metanal: Metaanal: Systematic review or overview Overview (careful) Quantitative review or overview Methodologic: review or overview Heterogeniety Homogeniety Medline Psychinfo Psycinfo Embase
Food for Thought Sometimes a rare condition that is being written up as a case report will include a substantial amount of background material. Don’t necessarily cross out a case-report if you are desperate for review type material.
Alternative Sources PubMed Clinical Queries—Review articles Cochrane—note only for RCTS –Cochrane reviews –Dare –Clinical Trials DARE (Database of Reviews of Effectiveness— librarian run project) Campbell Collaboration TRIP database—one stop shopping
Systematic Reviews Quiz 1 Systematic reviews are the same as ordinary reviews only bigger? Based on: Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001;322:98-101.
Systematic Reviews Quiz 3Only medical topics can be studied in systematic reviews. –More money on schools improved educational outcomes? –Do men or women make better leaders? –Does the sexual orientation of the parent matter to the child’s well being? –Are fathers more likely to treat their children differently than mothers? –Is job absenteeism an indicator of job dissatisfaction? –Are jurors influenced by the race of the defendants? –Is there a relation between poverty and violence?
Systematic Reviews Quiz 4Systematic reviews can be done well without experienced information personnel or good library support.
Systematic Reviews Quiz 5Systematic reviews have no relevance to the real world