Using the Internet to Answer Clinical Questions STFM Spring Conference - 2007 Brian S. Alper, MD, MSPH Anne-Marie Lozeau, MD, MS Beth Potter, MD

Disclosure “A funny thing happened to me on the way to rural family practice…” Dr. Alper is Editor-in-Chief of DynaMed™ and Medical Director of EBSCO Publishing

Learning Objectives Recognize that using the Internet to meet clinical information needs is practical Meet many useful web sites Learn efficient strategies for selecting useful resources for specific needs Use a free Internet portal to organize Internet use during patient care

Our Goal in Medicine Provide the best care… Provide patients the best information to guide health care decisions… Improve health outcomes… …based on the “truth” – separating medical knowledge from folklore

How do we know medicine? WE = society MEDICINE = clinical knowledge Scientific investigation Original research published in journals Systematic reviews EVIDENCE-BASED MEDICINE

How do we know medicine? WE = individual clinicians MEDICINE = clinical knowledge Consultants Colleagues Lectures Textbooks Rounds Precepting Guidelines CME Experts Experience PRACTICAL choices selected for efficiency

“Just-in-Case” vs. “Just-In-Time” Apprentice-style Memorization Soak up information now Retrieve information during practice Knowledge Mastery Find information when you need it Adult learning Problem-oriented Clinical application

Using Traditional Textbooks Not always written for clinical practice May not provide best research evidence (Are opinions well supported, or selectively supported?) May be years out-of-date Finding specific information within the text can be time-consuming

Using Guidelines Not always written for my clinical practice May not provide best research evidence (Are opinions well supported, or selectively supported?) May be years out-of-date Finding specific information within the text can be time-consuming May be unavailable for specific question

Using Journal Articles Individual journal articles may not relate to specific information need may not provide complete picture Finding one journal article can be time-consuming, let alone finding all the relevant articles Articles are written to promote research findings, not often written for clinical application BUT THIS IS OUR EVIDENCE SOURCE

Clinical Questions are Common Typical primary care physician has 2 questions for every 3 outpatient visits 3 questions for every 1 inpatient admission Ideal: Answer these questions with the best available evidence.

Previous research on answering clinical questions MEDLINE searches can answer clinical questions from primary care physicians 43% and 46% questions answered by medical librarian searches Mean search times were 27 minutes and 43 minutes J Fam Pract. 1996;43:140-144. Bull Med Libr Assoc. 1994;82:140-146.

Barriers to Finding Best Evidence 6 most important obstacles to answering doctor’s questions with evidence Time required to find information Difficulty modifying original question Difficulty selecting optimal search strategy Failure of resource to cover the topic Uncertainty when all relevant evidence found Inadequate synthesis of multiple bits of evidence BMJ 2002 Mar 23;324(7339):710

Result Busy clinicians use references considered Fast Likely to provide answers PREFERRED over evidence databases By family physicians (BMJ. 1999;319:358-361.) By family physician residents (Fam Med. 2003;35:257-260.)

The Need An evidence database must be: Fast Able to answer most questions To make EBM feasible in routine practice.

Evidence-Based Medicine Definition: Integration of best research evidence with clinical expertise and patient values. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine. How to Practice and Teach EBM. 2nd ed. London: Harcourt Publishers Ltd. 2000. p. 1.

Best Research Evidence Comprehensive – evidence only known to be best if all the available evidence is known Valid – critical appraisal determines potential for bias Systematic – selection and evaluation of evidence by protocol reduces author bias, investigator bias Current – every day new evidence could be best Synthesized – one study vs. the whole picture

EBM is not cookbook medicine “I know a lot, therefore I am” (replaceable by a computer) “I think, therefore I am” (never replaceable by computer)

EBM is not cookbook medicine Evidence does not directly advise individual patients Inclusion and exclusion criteria Co-morbidities Clinicians can use evidence to inform care of individual patients Individual patient values differ Balance of risks and benefits vary with individual

Ideal Sources for Clinicians Comprehensive – covering all relevant articles AND information needs. Valid/Systematic – driven by best available evidence. Relevant – focus on clinically relevant information. Easy to use – organized for use by clinicians. Affordable

Validity (“Truth”) Consider validity of original research PLUS validity of translation Was the best evidence identified? Was the best evidence selected? Was the evidence evaluated well? Is the summary accurate? Are conclusions based on the underlying evidence?

Evidence-Based = conclusions based on best available evidence “Evidence-based clinical reference” requires the following: Systematically identifying the evidence Systematically selecting the best available evidence from that identified Systematically evaluating the selected evidence (critical appraisal) Objectively reporting the relevant findings and quality of the evidence Synthesizing multiple evidence reports Deriving overall conclusions and recommendations from the evidence synthesis Changing the conclusions when new evidence alters the best available evidence

Systematic Evidence Identification Systematic Literature Search Commonly called Systematic Review Start with question, search for best available answers Systematic Literature Surveillance Also called Research Monitoring Start with literature being published, determine how best to use

Systematic Literature Search Start with clinical question Define inclusion criteria Search for all relevant information Assess methodology (validity) Combine results Determine clinical conclusion Apply to practice

Using Systematic Reviews to Answer Clinical Questions Cochrane Reviews may be “gold standard” when available and current many sources use Cochrane reviews Other Reviews - check whether process (search, evaluation) is systematic

Using Systematic Reviews to Answer Clinical Questions More difficult for non-RCT answers Consider additional info Most valid method for answering an individual question Not adequate to meet most information needs

Systematic Literature Surveillance Start with defined literature set Define inclusion criteria Select all relevant information Assess methodology (validity) Summarize results Determine clinical conclusion

Systematic Literature Surveillance - Uses Alert clinicians to critical info (EBM article summaries) Catalog for future searches (EBM search engine) Update clinical reference information (Integrated EBM)

Systematic Literature Surveillance - Advantages Cover all types of information - Diagnosis - Treatment - Etiology - Prognosis Keep up with current research Address current questions

Systematic Literature Surveillance - Disadvantages Can miss the big picture 1 study may be “rebuttal” “ping-pong” between sources overemphasis of study results Need to read collection of studies Miss older research (unless retrospective literature surveillance)

Ideal Sources Comprehensive – covering all relevant articles AND information needs. Valid/Systematic – driven by best available evidence. Relevant – focus on clinically relevant information. Easy to use – organized for clinician uses. USE SYSTEMATIC REVIEWS AND SYSTEMATIC LITERATURE SURVEILLANCE

Useful Links for Patient Care Internet portal: “Dr. Alper’s links” www.myHQ.com/public/a/l/alper Links to all resources in this presentation Organized for effective searches A starting point for use during patient care You can make your own! www.myhq.com

Internet Portals: Useful Links for Patient Care Organized for effective searches A starting point for use during patient care Access from any computer with internet connection Links to all resources in this presentation You can make your own!

Examples of Portals My HQ: www.myhq.com “Dr Alper’s links” www.myhq.com/public/a/l/alper I keep bookmarks www.ikeepbookmarks.com

* Asterisk - Requires Registration - MAY Require $$ Subscription

EVIDENCE-BASED MEDICINE Highly Valid EBM sources Relevant - many have direct clinician input FIRST CHOICE if fast enough (many are) FIRST CHOICE when research likely Quality research is possible? Research is present? (difficult to guess)

Clinical Scenario 52-year old male presents with Shingles (zoster) for 2 days. Clinical Concerns: short-term symptom relief prevention of Post-Herpetic Neuralgia

Evidence-based Search Engines Include systematic reviews Add other useful sources to increase chances of finding information where systematic reviews unavailable “medical content” searching instead of using Google or Yahoo

Using EBM Search Engines Examples: TRIP, InfoRetriever Broader than Systematic Reviews alone Limitations: individual article summaries may miss the “big picture” may have to read multiple items No subject overview

TRIP Database Turning Research Into Practice Identifies free websites with EBM and other medical content Catalogs titles and URLs to each page Inclusion based on Relevance and Rigor 300+ new articles added monthly Searches synonyms, cues misspelling

TRIP Database EBM Resources: systematic reviews, CATs, journal clubs Clinical Guidelines Query-answering Medical Images eTextbooks: background information Patient Information Journals – automatically searches PubMed with EBM filters

TRIP Database Comprehensiveness: high Validity: variable Relevance: moderate Convenience: easy to use Affordability: free

InfoRetriever POEMs Cochrane Abstracts Clinical Guidelines Clinical Decision Rules Diagnostic Test Calculators H&P calculators ICD-9 search tool 5-Minute Clinical Consult

InfoRetriever Comprehensiveness: moderate Validity: variable High for most referenced content Low for 5-Minute Clinical Consult Relevance: high Convenience: easy to use Affordability: $249/year

Clinical Evidence Highlights 230 common clinical topics Lists treatments for those topics Qualitatively describes Systematic Reviews and RCT data for and each treatment Compiles results for easy use Includes relevant background information Updates searches every 12 months Peer Reviewed

Key sources for Systematic Reviews Cochrane Library DARE Major benefit: best evidence (at time of review) Major limitation: limited scope of coverage Clinical Evidence AHRQ

Clinical Evidence Comprehensiveness: modest Validity: high - limited to RCTs Relevance: high - clinical focus Convenience: easy to use Affordability: $239/year ($120 students)

PubMed Clinical Queries Use search filters that have been researched by Haynes, et al. who are involved with the McMasters EBM group Help to refine EBM searches in Pubmed

Clinical Queries Comprehensiveness: moderate Validity: High Relevance: Low Often finds many articles Still need to sift through a lot of articles Convenience: takes practice New feature allows you to receive updates when new info is added to your search Affordability: Free

National Guideline Clearinghouse Guidelines search engine FREE - www.guideline.gov Links available in other sources Many guidelines are not “evidence-based”, even when labeled “evidence-based” Levels of evidence used vary widely across guidelines

National Guidelines Clearinghouse Comprehensiveness: moderate Validity: Low to High Not all guidelines are EBM Some guidelines are expert -based Relevance: Low Often finds many guidelines Convenience: takes practice Affordability: Free

DynaMed Nearly 2,000 topics combining best available evidence with clinical focus Updated continuously with systematic literature surveillance > 500 Journals directly or via multiple journal review services Cochrane reviews National Guideline Clearinghouse Template system for rapid browsing

DynaMed Peer Review in process Includes ICD-9 Codes and patient handout links for each topic Easy to navigate Alphabetical topic browse Standardized templates Some research-heavy topics can get difficult to navigate UNTIL new topics created

DynaMed Comprehensiveness: shown to answer 69% of clinical questions in primary care in 2004 Validity: variable & explicit Relevance: high, clinical focus Convenience: easy to use Affordability: $350/year or effort ($150 residents, $100 students)