4Levels of References Primary: Peer Reviewed Secondary: Not peer-reviewedPrimaryWhen someone says Primary Literature, they usually mean something that is peer reviewed, like a journal article published in a peer reviewed journal or a textbook from a highly respected author who quotes from and cites peer reviewed articles.The secondary level of references or literature refers to things like newspaper articles, authoritative web sites, and such that are usually opinions or reports of occurrences, news, surveys, and such. Also included in secondary sites are databases like the ones at the CDC which are statistics collected from state and local agencies.Secondary
5Levels of Peer Reviewed Information Primary: original researchSecondary: review articlesTertiary: textbooks, summariesPrimary Information Source:Articles in scientific journals that report the findings from research studies and experiments are considered "primary" information sources, because they are "first hand" accounts of the actual research and the results, i.e. the original scientific evidence. Papers about original empirical research must conform to acceptable scientific standards for conducting and reporting such research, and must pass peer-review to be published as articles in a scientific journalSecondary Information Source:The various original scientific studies described above generate a wealth of "primary evidence" that periodically needs to be gathered, organized, and assessed by the clinical and scientific communities to give us a sense of the current "state of the science" on a given topic. These types of research "Review" articles are considered a secondary source because they are "second-hand" accounts which integrate the scientific evidence from a number of original research studies. However, such scientific "Review" articles must also conform to scientifically acceptable methodologic standards for gathering, synthesizing, and reporting the available evidence, and also must be peer-reviewed before being published in a reputable scientific journal.Tertiary Information Sources:Textbooks offer an example of a usually reliable source of information, that may or may not be peer-reviewed. The more accurate and reliable textbooks will contain information that is drawn only from the evidence from primary and secondary source articles that have been published in the peer-reviewed scientific literature, and that conform to scientific standards for summarizing and interpreting that information. For instance, the author(s) of an accurate and reliable textbook will appropriately and objectively summarize all information available on a topic, and not just "pick and choose" certain evidence or interpret the selected evidence in a biased manner. In accredited institutions, textbooks are reviewed and approved by the faculty of the institution, which provides another level of "peer-review" of the information in the textbook. Textbooks in wide use by a number of accredited institutions may give a further indication of the extent to which educators, clinicians, and scientists consider the book a credible and legitimate source of accurate and complete informationPrimarySecondaryTertiary
6Primary Peer Reviewed Literature Primary Source Evidence:The second table (Slide 6) titled "Continuum of Study Designs and their Causal Inference" demonstrates the hierarchy of the different types of research studies. This hierarchy is based on the ability of the research study design to satisfy the "Criteria for Causality" (some of which were identified in the previous slide). The research examples at the bottom of the table (e.g. case studies) are good for beginning the process of developing cause-and-effect research hypotheses, but such studies can only provide very weak evidence since they don't meet many of the Criteria for establishing Causality. For example, in testing a new angina drug, viagra, there was this interesting side effect. Moving from the bottom to the top of the pyramid, the rigor of the research studies is increased, more of the causal criteria are met, and a stronger case may be made for drawing conclusions (inferences) about cause-and-effect. At the top of the triangle is the experimental randomized controlled trial (RCT), which is the strongest study design for meeting the causal criteria, and therefore is the "gold standard" of scientific evidence.
7Secondary Sources Secondary Source Summaries of Evidence: The "primary evidence" from the research studies described above is periodically "Reviewed" by gathering all of the evidence available on a given topic, and comparing and contrasting the various research studies and the results from those studies. Review articles follow acceptable scientific methods which again make use of the common-sense "Criteria for Causality" to compare and contrast the strength of the different studies for supporting (or refuting) a hypothesized cause-and-effect relationship. Generally, Review articles fall into one of 3 broad categories in order of increasing scientific rigor: the narrative review, the systematic review, and meta-analyses, which are briefly described below.Narrative Reviews are essentially "essays" wherein the researcher/author focuses on a particular clinical topic, and presents in their opinion what they feel are some of the highlights worth noting from the scientific literature they review. Narrative Reviews are the weakest of the "Secondary Source" research literature, since the methods and the reporting are somewhat subjective, and therefore may be more prone to bias. Articles that report original empirical research studies (described as "Primary Source" evidence above) typically also contain good "narrative overviews" of the "existing evidence" as an introduction and background to the article, to explain and justify why the original research study was conducted.Systematic Reviews are conducted according to scientific methodologies that are explicit, explicitly stated, and reproducible. For instance, the methods section of a systematic review will describe the literature search in enough detail (e.g. what databases were searched, for what years, search terms employed), that the search strategy could be replicated by another researcher and produce essentially the same yield of literature. The "primary evidence" articles are then systematically and explicitly compared and contrasted along a number of important dimensions of research designs that reflect the "Criteria for Causality", which makes for a stronger and more useful "review of the resulting evidence" than do simple narrative reviews. This "attention to detail" in conducting and reporting the systematic review, helps the readers of the review article (i.e. the clinical and scientific communities) to determine how strong the accumulated evidence appears to be, without relying solely on the opinion of the author.Meta-Analyses take the systematic review to the highest and most rigorous level of integrating "primary evidence" research, by statistically synthesizing the quantitative results across similar studies. Essentially, meta-analyses pool the data and results from a number of original research studies, with the intent of producing a more precise summary of the existing evidence for (or against) a cause-and-effect relationship. For example, a meta-analysis may collect results from a number of epidemiologic studies, and statistically combine the data to produce a more precise overall "risk estimate" of the relationship between the causative risk factor and the effect (the disease).
9Centers for Disease Control and Prevention (CDC) : www.cdc.gov Description The CDC is the site to visit if you are looking for information on chronic diseases,injuries and disabilities, or guidelines on their prevention.In addition to these sources, the CDC has valuable data on travellers’ health, plus full-text access to the Morbidity and Mortality Weekly Report (MMWR).Cost FreeAnalysis The stated aim of the CDC is to ‘promote health and quality of life by preventing and controlling disease, injury and disability. In addressing this broad agenda the CDC has developednumerous information resources of use and interest to all health professionals.Fortunately, many of the resources held at CDC can be accessed through one sophisticated search tool, known asCDC WONDER :
10CDC WONDERCDC WONDER provides query access to about 40 text and numerical databases. For example,When you select the mortality data set, to retrieve data on the causes of death, you can define your search parameters to include a particular state, race, gender or age. Text databases such as the Prevention Guidelines and the MMWR can be searched by keyword.For information about travellers’ health, CDC provide a hypertext map of the world. By pointing and clicking your mouse on a particular country you can identify its current vaccine requirements, details of any prevalent diseases, plus general health and travel advice. This type of information is updated on a regular basis to take account of recently reported disease outbreaks.
11Internet Mental Health www.mentalhealth.com Description A one-stop source for authoritative information on mental illness.Cost FreeAnalysis Internet Mental Health describes itself as a ‘free encyclopedia of mental health’.Detailed here is information on the most prevalent mental disorders, supplementedwith the most common psychiatric medications.For every mental disorder discussed, visitors to the site can find a description of the illness, read a synthesis of recent research into its diagnosis, treatment and cause, and be directed to information booklets compiled by professional organizations and support groups.Drug information at this site includes details on adverse effects, contraindications and dosages.The site also enables you to perform an online diagnosis of a number of disorders including anxiety, eating, mood, personality and those related to substance abuse. Diagnosis is based on the results of a questionnaire you complete while online. Although only a qualified professional can accurately diagnose a mental disorder its use as a teaching aid may be considerable.
13National Institutes of Health (NIH): www.nih.gov Description Recognised as one of the world’s foremost biomedical research centres, theNational Institutes of Health provide a range of invaluable sources of information for healthprofessionals.Cost FreeAnalysis The mission of the NIH is to ‘uncover new knowledge that will lead to better health foreveryone’. It works towards this mission by conducting its own research and supporting theresearch of scientists throughout the world [Its budget for 1998 was $13.6 billion].The Web site has links to many health information resources that come under the NIH umbrella.Some of the key links include:• The Consensus Development Program. The Consensus Statements and the TechnologyAssessment Statements that are authored under this program are authoritative guides tocurrent medical issues. Scanning the titles of some of the most recent Statements –‘Effective medical treatment of opiate addiction’ and ‘Acupuncture’ – demonstrates thecommitment of the NIH to focus on clinical issues.• CancerNet. Developed by the National Cancer Institute (NCI), CancerNet provides factsheets on topics such as cancer detection, prevention and therapy. It is also possible toperform a literature search on the CancerNet database to identify references to publishedjournal articles, conference proceedings, government reports and monographs that relate tocancer.
14OncoLink: www.cancer.med.upenn.edu Description Detailed and authoritative information on all aspects of cancer.Cost FreeAnalysis This site is the only one within this list that is disease specific. Its inclusion, however, reflects the quality of the information and the ease with which cancer-related questions canbe answered.A clear and simple front menu allows you to approach the resources at OncoLink by 10 main categories, including disease, medical specialty (chemotherapy, surgical oncology etc.) and the causes of cancer.For cancer patients and their families the ‘Psychological Support Menu’ provides links to documents that discuss how one can cope with cancer, as well as specific issues such as ‘Coping with grief and loss’.In contrast, physicians undertaking research on cancer can link to the ‘Clinical Trials’ section. Here, all trials currently being conducted by the National Cancer Institute and the University of Pennsylvania are cited and discussed.If the subject you are searching for does not lend itself to this menu approach, the OncoLink search engine can be used. As this uses AltaVista technology searching is both familiar and very fast. A search for +malignant +melanoma +sun, for example, produces a ranked list of> 55 relevant documents.Perhaps the most remarkable aspect of the OncoLink Web site is that, despite the mass of information held here, it is very easy to find what you are looking for.
15PubMed – MEDLINE on the Web www. PubMed.ncbi.nlm.nih.govDescription PubMed, developed by the National Library of Medicine, provides a free WorldWide Web interface to MEDLINE, the world’s premier biomedical database. Dating from 1966,the MEDLINE database currently contains just more than 10 million bibliographic citations drawnfrom around 5000 biomedical journals.Cost FreeAnalysis Free Internet access to MEDLINE is a relatively new service but one which an increasing number of Web sites now offer.An annotated list of free MEDLINE sites is available at:SPages/ medline.aspThough all these services have unique and useful characteristics, it is considered that the PubMed MEDLINE to be the best version currently available.The strengths of this service are:• Currency. By incorporating data from the Pre-MEDLINE file, citations appear in the databasefar more quickly than in other versions of MEDLINE. Some measure of the currency ofPubMed can be gauged by the observations showing that articles published in journals suchas JAMA and the New England Journal of Medicine are indexed – and thus searchable –within one week of publication; articles from the BMJ and Lancet appear within a fortnightof publication.
16Pubmed(cont.)• Powerful searching. Using pulldown menus it is possible to search specific MEDLINE fields(author name, MeSH term, title word, etc.) and use the Boolean operators (AND, OR and NOT)to combine search terms. PubMed also has a ‘Clinical Query Filter’ that allows you to restricta search to one of different study categories. Once a search has been run and citationsselected, PubMed also gives you the option to find other related articles.• Links to full-text articles. The NLM is currently establishing links with variouspublishers to enable MEDLINE searchers to link directly to the full text of a givenarticle. (Note: This feature is publisher specific and users may incur a fee to seethe full text of an article.)• Dynamic links with related databases. A number of MEDLINE articles have linksto related databases, such as the Online Mendelian Inheritance in Man (OMIM)database and the Molecular Modelling Database (MMDB).
19Reuters Health: www.reutershealth.com Description A health and medical daily news service keeping health professionals abreast ofhealth stories and scares.Cost [may be charged].Analysis Covering around 50 news stories per day, Reuters Health News is the most comprehensive medical news service on the Internet.To help health professionals find stories most pertinent to their interests, articles are organised into a number of discrete categories, such as clinical, epidemiology and public health. Alternatively, relevant current and archive stories can be identified using a powerful search interface.All news items cite the original source of the story – typically a research item in a medical journal – and if a story contains a reference to a particular drug, there will be a hypertext link to Reuters Clinical Pharmacology database, where additional information can be found.Although this is a subscription-based service, visitors can access the consumer health stories and a number of items from the medical news section every day, without charge. Searching facilities, however, are restricted to subscribers.Free Alternative The Medical News pages of the Doctor’s Guide to the Internet:andhave a reasonably comprehensive news coverage which, if you wish, can be sent to yourelectronic mailbox on a regular basis. Searching of the news archive is also possible at this site.
21RxList – the Internet drug index www.rxlist.com Description A database of prescription pharmaceuticals.Cost FreeAnalysis RxList, the ‘Internet drug index’, provides information on prescriptiondrugs. Drug information can be found by either searching the database – by brand or generic name, imprint code or drug category – or by browsing the ‘Top 200’ prescription drugs.
22RxList (Cont.)When searching by brand, US names as well as ‘foreign’ ones can be searched for.Basic prescribing data is supplemented with 300 RxList monographs. These detailed reports describe key features such as contraindications, adverse effects and drug interactions.
24TRIP – Turning Research into Practice www. gwent. nhs. gov TRIP – Turning Research into Practice trip/test-search.htmlDescription A searchable index to the premier evidence-based medicine (EBM) resources.Cost FreeAnalysis One of the problems facing the clinician who wishes to practice evidence-based medicine (EBM) is the disparate way research findings are disseminated. Although the CochraneDatabase (http://www.cochrane.co.uk/abstracts) is the best source for identifying the results of systematic reviews, other sources, such as clinical practice guidelines and briefing papers, may also be of interest. Finding this information can, however, be extremely time-consuming.In recognition of this, the Primary Care Clinical Effectiveness Team for Gwent have created the TRIP database, a single searchable index to a range of EBM resources. Sources indexed by TRIP include the Cochrane Database of Systematic Reviews, Canadian Clinical Practice Guidelines Infobase, Database of Abstracts of Reviews of Effectiveness (DARE) and the evidence-based health-care journals Bandolier and Evidence Based Medicine.
27WebMedLit: www.webmedlit.com Description WebMedLit provides health professionals with a way of keeping up to date with research published in a number of quality medical journals, including BMJ, JAMA and the NEJM.Cost FreeAnalysis Describing itself as a ‘medical headlines service’, WebMedLit scans the Web every day for updates to the medical titles it is currently tracking. For example, the BMJ, published on a Friday, is picked up by this service and is indexed and searchable by Saturday morning.Articles remain in the WebMed-Lit database for around 6 weeks. After this time they should be retrievable at the PubMed MEDLINE site.What differentiates WebMedLit from the countless other Web sites providing links to medical journals is its citation database. This gives users the opportunity to search the contents of these top medical journals from one source, and the functionality to link directly back to the article (or abstract) at the individual journal’s Web site.As an alternative to searching the WebMedLit database, the latest medical literature can be viewed by subject. Ten general subjects have been defined, including AIDS, cardiology, and cancer.
30World Health Organization www.who.int Description Details of WHO programmes (Global Programme for Aids, Global Programme for Vaccine and Immunisation, etc.); database of WHO publications; statistical databases; Weekly Epidemiological Report; press releases and newsletters.Cost FreeAnalysis In recognition of the importance of the Internet, the WHO has made the Weekly Epidemiological Report available via the World Wide Web. This electronic journal provides health professionals with a means of obtaining rapid and accurate epidemiological information,along with details of new disease outbreaks.To identify subject-specific WHO resolutions, guidelines and journal articles, the WHO Management Information (WHOMIS) can be searched.The WHO also provides a statistical service (WHOSIS) giving Internet users access to a variety of data sources, such as WHO Global Health for All Indicators Database and the Global Programme for Vaccines and Immunization Incidence data.
33What is MEDLINE? >5,000 biomedical peer reviewed journals Old Index Medicus databaseNational Library of Medicine (NLM)>12 million citationsBack to mid 1960s
34Where is MEDLINE? Pubmed (NLM) OVID MD Consult etc Links to Full text journal articlesBooksEBM databasesYes, it is true. MEDLINE is all over the place. Selecting the search engine to go with the search topic is an important decision. OVID provides a powerful search engine, allowing for quick access to MeSH, command searching, cross-file searching with EBM databases, and links to MEDLINE in context to references and other full-text articles within the OVID selection of 82 electronic journals. MD Consult provides one-stop shopping with reference and text books, current articles patients are reading, web resources, and over 40 electronic journals. (The search engine for MD Consult is not as good as OVID’s search engine.) Pubmed includes Biomed Central journals and a feature called Linkout. Once activated at FSU, approximately 1,000 electronic journal titles will be linked to specific references. Science Direct links MEDLINE references to a large group of FSU-subscribed e-journals published by Elsevier. Cambridge Scientific is another FSU-subscribed vendor but I can’t recommend it for MEDLINE searching. Medscape is a very popular Internet site with a broad range of free literature, although much of the full-text information is not peer reviewed. I could go on. I won’t!
35MEDLINE Fields Author Title Date Type of publication Journal, Volume, IssuePage numbersAbstractInstitutionMeSH HeadingsA lot of cataloging numbersetc
36What’s a MeSH Heading? "Medical Subject Headings" vocabulary terms from subject analysis of literatureindexing journal articles for Index Medicus® and MEDLINEcataloging books and audiovisualscontrolled vocabularyarranged in a hierarchical MeSH Tree Structuresupdated regularly
48Tricks Find one good article Look at MeSH headings Find similar articlesUse Related Articles in PubMedSearch same MeSH terms
49Too Many Hits: Increase Specificity Use more specific terms in free text searchUse mesh search rather than text word searchCombine searchesUse more specific mesh headingsAdd terms (using AND)Limitlanguage of articlehuman or animal subjectspublication types (e.g. randomized controlled trials, reviews, etc.)Type of the studyyear of publicationAge groupetcWhen you get too many hits you will need to Increase Specificity bynarrowing your questionusing more specific terms in Free Text searchusing a Subject search rather than Textword searchusing more specific Subject headingsselecting specific subheadings with Subject MeSH headingsadding in terms (using AND) to represent other aspects of the questionlimiting to language of article, to human or animal subjects, to publication types (e.g. randomised controlled trials, reviews, etc.), to country or year of publication.
50Too Few Hits Increase Sensitivity Find more search terms from good articlesTry different combinations of termsUse truncation (*) or wildcard (?) in Text word OR SubjectUse OR to add and combine termsUse NEAR to retrieve same sentence items, either orderUse combination of Textword and Subject searchesSelect All Subheadings with Subject MeSh termsSearch further back in timeWhen you find nothing or just a few articles you need to Increase sensitivity by:broadening your question!finding more search terms from relevant recordstrying different cobinations of termsusing truncation (* or $) or wildcard (?) in Free Text/Textword OR Thesaurus/Subjectadding in and combining terms of related meaning using ORusing NEAR to retrieve items in the same sentence in either orderusing a combination of Free Text/Textword and Thesaurus/Subject searchesusing explosion feature of Thesaurus searchesselecting All Subheadings with Thesaurus/Subject MeSh termssearching further back in time
51Find Good Articles Recent Applicable Relevant Good research criteria Not just the first full text articles you find
56Types of Health Statistics Questions Fact lookupsResearchPresentationsSocial and Policy indicatorsetcFor Fact Lookups, you need to find a table with the answers you need. One should understand the source of the information like the Census bureau or CDC while relating it to the definitions and knowing its limits. Look for a prepared table of digested data in published reports.When doing research or a presentation you need information to form the bases of comparisons. In this case you will need to consider which population you are talking about and what question you are answering. talking about and what question you are answering. You may need both journal articles and the original data.For Health and social policy, you need statistical indicators of progress. Uses can include drafting policies or legistation, monitoring success of health care and measuring the impact on society.Health indicator can includeQuality of life indicatorsPublic health progressQuality of careEvaluation of effortsYou will need literature that summarize results and table that focus on key points.
57Internet Portals of Health Stats Lists of links that provide starting points for browsing or searchingKeyword search in portal vs GoogleGeneral idea what you wantThe Related Health Services Research Web SitesThe NCHS portal:For links to portals, go to Related Health Services Research web sites. The Data Sets and Health Statistics sections are particularly relevant.Go to the NCHS site and explore the variety of statistical resources available. What parts of this site are useful for fact lookup, research and indicators?
58Other Statistical Web Sites CDC Data and StatisticsFedStats Home PageCompare these twoU Michigan’s Statistical Resources on the WEB – HEALTHWhat type of stats
59Lexis-Nexis Statistical Universe Subscription resourceSearches stat dataSubject ListLimit searchReports or tablesLet’s try it out. Notice you can limit by type of data We can look for reports about tobacco regulations
60MMWR Morbidity – illness Mortality – death http://www.cdc.gov/mmwr/ Disease TrendsTables - searchableThe morbidity and mortality weekly report is based on data collected by government agencies. Mortality data is based on death certificates at the National Vital Statistics System. Morbidity data is based on surveys and reports of outbreaks that funnel into the CDC. Only reportable diseases are close to accurate. Wellness data is not available because it is not collected.In the NCSR you can find the whole report on deaths in some year.
61Health Care Data Healthcare Cost and Utilization Project HCUPnet Hospital dischargesAmbulatory serviceCostsAmount of careBy diagnosis and procedureSurveys of hosp, physicians, nursing homesSee if you can find the median cost, length of stay and hospital death rate associated with hosp treatment of acute myocardial infarction.
62Health Consequences Costs to society, individuals Cost from care Costs of illnessImpact on infrastructureCenter for Medicare and Medicaid Services (formerly HCFA) site has all this information
63State and International Data Floridahealthstat.com - Where Florida Health Data ResidesDOH EpidemiologyKFF State Health Facts OnlineUnited Nations Statistics DivisionWorld Health Organization Research Tools
64Create or Open Library Create new Open Save in My Documents File Menu Name itPick a folderOpenShould pop upSave in My Documents
65How to Populate the Library Type in referencesConnect to Pubmed or other database and captureSearch Pubmed or other DB and save results then importIn WOS or OVID, search and export directly to EndnoteEndnote Libraryof References
66How to Type in Citation In Endnote Pick reference type Type in fields References MenuSelect New…Pick reference typeType in fieldsClose the window
67How to Capture References Endnote Tools MenuChoose Connect…Pick a sourceMEDLINE -> Pubmed at NLMIf OVID, ignore userID and PW, hit OK
68Perform Search If article known Use Author and Title Perform Search Pick articleCopy Reference to...Your Library
69Cite While You Write Cursor on spot In Endnotes Select reference With Library openSelect referenceTools MenuCite While You WriteInsert Selected CitationCitation and reference appearEndnote Toolbar in Word
71CONCLUSIONThe only way to fully appreciate and exploit the range of resources described in this session is to visit these sites and explore.In particular, it is important to pay special attention to the ‘What’s New’ links. Although at times these may simply serve to distract you from your original objective, on other occasions they will bring to your attention new and useful resources which otherwise would have remained undiscovered.As you trawl the Internet you will undoubtedly discover numerous other sites of interest and effectively compile your own ‘top ten resources’.
72Questions ? Comments?!Thank you for your attention!