Using the Internet to Answer Clinical Questions STFM Spring Conference Brian S. Alper, MD, MSPH Anne-Marie Lozeau, MD, MS Beth Potter, MD

Disclosure “A funny thing happened to me on the way to rural family practice…” Dr. Alper is Editor-in-Chief of DynaMed™ and Medical Director of EBSCO Publishing

Learning Objectives Recognize that using the Internet to meet clinical information needs is practical Meet many useful web sites Learn efficient strategies for selecting useful resources for specific needs Use a free Internet portal to organize Internet use during patient care

Our Goal in Medicine Provide the best care… Provide patients the best information to guide health care decisions… Improve health outcomes… …based on the “truth” – separating medical knowledge from folklore

How do we know medicine? WE = society MEDICINE = clinical knowledge Scientific investigation Original research published in journals Systematic reviews EVIDENCE-BASED MEDICINE

How do we know medicine? WE = individual clinicians MEDICINE = clinical knowledge Consultants Colleagues Lectures Textbooks Rounds Precepting Guidelines CME Experts Experience PRACTICAL choices selected for efficiency

“Just-in-Case” vs. “Just-In-Time” Apprentice-styleMemorization Soak up information now Retrieve information during practice Knowledge Mastery Find information when you need it Adult learning –Problem-oriented –Clinical application

Using Traditional Textbooks Not always written for clinical practice May not provide best research evidence (Are opinions well supported, or selectively supported?) May be years out-of-date Finding specific information within the text can be time-consuming

Using Guidelines Not always written for my clinical practice May not provide best research evidence (Are opinions well supported, or selectively supported?) May be years out-of-date Finding specific information within the text can be time-consuming May be unavailable for specific question

Using Journal Articles Individual journal articles –may not relate to specific information need –may not provide complete picture Finding one journal article can be time- consuming, let alone finding all the relevant articles Articles are written to promote research findings, not often written for clinical application BUT THIS IS OUR EVIDENCE SOURCE

Clinical Questions are Common Typical primary care physician has –2 questions for every 3 outpatient visits –3 questions for every 1 inpatient admission Ideal: Answer these questions with the best available evidence.

Previous research on answering clinical questions MEDLINE searches can answer clinical questions from primary care physicians –43% and 46% questions answered by medical librarian searches –Mean search times were 27 minutes and 43 minutes J Fam Pract. 1996;43: Bull Med Libr Assoc. 1994;82:

Barriers to Finding Best Evidence 6 most important obstacles to answering doctor’s questions with evidence –Time required to find information –Difficulty modifying original question –Difficulty selecting optimal search strategy –Failure of resource to cover the topic –Uncertainty when all relevant evidence found –Inadequate synthesis of multiple bits of evidence BMJ 2002 Mar 23;324(7339):710

Result Busy clinicians use references considered –Fast –Likely to provide answers PREFERRED over evidence databases By family physicians (BMJ. 1999;319: ) By family physician residents (Fam Med. 2003;35: )

The Need An evidence database must be: –Fast –Able to answer most questions To make EBM feasible in routine practice.

Evidence-Based Medicine Definition: Integration of best research evidence with clinical expertise and patient values. –Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine. How to Practice and Teach EBM. 2nd ed. London: Harcourt Publishers Ltd p. 1.

Best Research Evidence Comprehensive – evidence only known to be best if all the available evidence is known Valid – critical appraisal determines potential for bias Systematic – selection and evaluation of evidence by protocol reduces author bias, investigator bias Current – every day new evidence could be best Synthesized – one study vs. the whole picture

EBM is not cookbook medicine “I know a lot, therefore I am” (replaceable by a computer) “I think, therefore I am” (never replaceable by computer)

EBM is not cookbook medicine Evidence does not directly advise individual patients –Inclusion and exclusion criteria –Co-morbidities Clinicians can use evidence to inform care of individual patients –Individual patient values differ –Balance of risks and benefits vary with individual

Ideal Sources for Clinicians Comprehensive – covering all relevant articles AND information needs. Valid/Systematic – driven by best available evidence. Relevant – focus on clinically relevant information. Easy to use – organized for use by clinicians. Affordable

Validity (“Truth”) Consider validity of original research PLUS validity of translation –Was the best evidence identified? –Was the best evidence selected? –Was the evidence evaluated well? –Is the summary accurate? –Are conclusions based on the underlying evidence?

Evidence-Based = conclusions based on best available evidence “Evidence-based clinical reference” requires the following: 1. Systematically identifying the evidence 2. Systematically selecting the best available evidence from that identified 3. Systematically evaluating the selected evidence (critical appraisal) 4. Objectively reporting the relevant findings and quality of the evidence 5. Synthesizing multiple evidence reports 6. Deriving overall conclusions and recommendations from the evidence synthesis 7. Changing the conclusions when new evidence alters the best available evidence

Systematic Evidence Identification Systematic Literature Search Commonly called Systematic Review Start with question, search for best available answers Systematic Literature Surveillance Also called Research Monitoring Start with literature being published, determine how best to use

Systematic Literature Search Start with clinical question Define inclusion criteria Search for all relevant information Assess methodology (validity) Combine results Determine clinical conclusion Apply to practice

Using Systematic Reviews to Answer Clinical Questions Cochrane Reviews –may be “gold standard” when available and current –many sources use Cochrane reviews Other Reviews - check whether process (search, evaluation) is systematic

Using Systematic Reviews to Answer Clinical Questions More difficult for non-RCT answers Consider additional info Most valid method for answering an individual question Not adequate to meet most information needs

Systematic Literature Surveillance Start with defined literature set Define inclusion criteria Select all relevant information Assess methodology (validity) Summarize results Determine clinical conclusion

Systematic Literature Surveillance - Uses Alert clinicians to critical info (EBM article summaries) Catalog for future searches (EBM search engine) Update clinical reference information (Integrated EBM)

Systematic Literature Surveillance - Advantages Cover all types of information - Diagnosis - Treatment - Etiology - Prognosis Keep up with current research Address current questions

Systematic Literature Surveillance - Disadvantages Can miss the big picture –1 study may be “rebuttal” –“ping-pong” between sources – overemphasis of study results Need to read collection of studies Miss older research (unless retrospective literature surveillance)

Ideal Sources Comprehensive – covering all relevant articles AND information needs. Valid/Systematic – driven by best available evidence. Relevant – focus on clinically relevant information. Easy to use – organized for clinician uses. USE SYSTEMATIC REVIEWS AND SYSTEMATIC LITERATURE SURVEILLANCE

EVIDENCE-BASED MEDICINE Highly Valid EBM sources Relevant - many have direct clinician input FIRST CHOICE if fast enough (many are) FIRST CHOICE when research likely –Quality research is possible? –Research is present? (difficult to guess)

EBM in clinical practice Need to make EBM feasible in routine clinical practice. Internet Portals = useful links for answering questions quickly during patient care.

Internet Portals Connect you to useful links Ideal for use during patient care Organized for effective, quick searches Access from any computer with internet connection (mobile bookmarks/favorites) You can make your own!

Examples of Portals My HQ: I keep bookmarks: Google bookmarks: iGoogle:

Internet Portals myHq example: “Dr. Alper’s links” Links to all resources in this presentation Organized A starting point for use during patient care To make your own:

* Asterisk - Requires Registration - MAY Require $$ Subscription

Internet Portals I keep bookmarks:

Internet Portals iGoogle: Uses Google Bookmarks Can also function as an RSS Feed Reader

Clinical Scenario 32-year old female presents with 1 day history of Left facial paralysis. You diagnose Bell’s Palsy. Clinical Concerns: what is the best treatment? –Should I treat with oral steroids? –Would oral antiviral treatment be beneficial?

Where is my answer? EBM Search Engines Systematic Reviews Integrated EBM

Evidence-based Search Engines Include systematic reviews Add other useful sources to increase chances of finding information where systematic reviews are unavailable. “Medical content” searching vs. Google, Yahoo or other unfiltered information.

Using EBM Search Engines Examples: TRIP, Essential Evidence Plus (previously InfoRetriever) Broader than Systematic Reviews alone Limitations: –Individual article summaries may miss the “big picture” –May have to read multiple items –No subject overview (vs. integrated EBM)

TRIP Database Turning Research Into Practice Identifies free websites with EBM and other medical content EBM Resources: systematic reviews, CATs, journal clubs, PubMed with filter Inclusion based on Relevance and Rigor Clinical guidelines, images, journals Searches synonyms, cues misspelling

Clinical Scenario 32-year old female presents with 1 day history of Left facial paralysis. You diagnose Bell’s Palsy. Clinical Concerns: what is the best treatment? –Should I treat with oral steroids? –Would oral antiviral treatment be beneficial?

TRIP Database Comprehensiveness: high Validity: variable Relevance: moderate Convenience: easy to use Affordability: free

BMJ Clinical Evidence Started in 1999 Review 2,500 therapeutic interventions Organized by topics Qualitatively describes Systematic reviews and RCT data Includes background information Updates every 12 months Peer Review Has a separate surveillance program – BMJ Updates

BMJ Clinical Evidence Comprehensiveness: Modest – limited topics Validity – High – Systematic reviews and RCTs Relevance – High – Clinically Focused Convience – Easy to Use Affordability – 12 months Subscription £137/€203/$260 (standard individual) £124/€184/$236 (10% discount for BMA members) £69/€102/$131 (student/nurse) £35/€52/$67 (50% discount for BMA student members)

PubMed Clinical Queries Use Search Filters that have been researched by Haynes, et al. who are involved with the McMasters EBM group

PubMed Clinical Queries Comprehensiveness: Moderate Validity: High Relevance: Low –Often finds many articles –Still need to sift through many articles Convenience: Takes practice –New feature allows you to receive updates on your searches Affordability: Free

Essential Evidence Plus (Formerly Inforetriever)

Essential Evidence Plus Comprehensiveness: Moderate Validity: High Relevance: Moderate –Often finds many articles –Still need to sift through many reviews Convenience: Takes practice Affordability: $79 for a one year subscription and daily poems

Integrated EBM Combining –Systematic literature surveillance –Systematic reviews –Single interface with integration (synthesis) of content

DynaMed Integrated EBM Resource combines best available evidence with clinical focus Updated continuously with systematic literature surveillance –> 500 journals directly and via 10 journal review services –Cochrane Reviews –National Guideline Clearinghouse Template system for rapid browsing

DynaMed Peer review in process Includes ICD-9/10 Codes and patient information handout links for each topic Easy to navigate –Alphabetical topic/category/text browse –Standardized templates –Some research-heavy topics can be difficult to navigate UNTIL new topics created

DynaMed Comprehensive - shown to answer 69% of clinical questions in primary care in 2004 Validity - variable & explicit Relevance - high, clinical focus Convenience - easy to use Affordable - $350/year or effort $150 resident or effort w/faculty $150 resident or effort w/faculty $100 student $100 student