1. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH. Terrie Wheeler, MLS Anne White-Olson, MLS Brigit Sullivan, MLS Tom Moskal, DVM Bibliometric Opportunities at The NIH Library

2. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Establishing linkages: using the literature to identify value Using the literature to assess science requires: 1.clear linkages from funding to papers to new science, and 2.clean data 3.meaningful metrics

3. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Concerns with standard publication metrics Impact Factor –Average # of citations (within one year) received by all papers in a journal published within the subsequent 2 years h-index –g-index –tapered h-index –a-index –etc. Source: http://en.wikipedia.org/wiki/File:H-index-en.svg

4. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Percentile Ranking Normalizes citation counts Is not as affected by a skewed distribution Provides information about publication impact Is calculated fairly cleanly Over time percentile ranking in a discipline is anticipated

5. Percentile ranking chart from TR’s Essential Science Indicators, 200Percentile ranking chart from TR’s Essential Science Indicators, 200

6. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Using the literature to identify value NIAID – impact on standards of care NIMH – impact of extramural funding CSR – seeking which study sections have funded most drug/device discoveries NHLBI – papers from funded research over the last ten years

7. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH NIDCD PIs and IC Collaborators Who are the NIH PIs that the NIDCD scientists collaborate with? How many publications have they authored during a specific time period (2007-2012) Who is first author? Who is last author? –What number author are they?

8. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH How to approach this? PubMed (few analytics) Web of Science (analytics, but fewer bib records than PubMed) BiomedExperts Elsevier NIH IRP Website NIDB Reporter

9. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Capturing the Data Web of Science Author Search and Analyze Results/Author Endnote, Each Author was individual custom group Excel: Each NIDCD author had spreadsheet, where listed all Authors and counts of what number author s/he was Exported Endnote to Word-Bibliographies for each author.

11. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Take Away Dirty Data and numbers vary per system Different paths give different numbers. Pick one and use that consistently No one system has all the answers Value of Librarians: Know the systems, –Know data, Ask a lot of questions, Use a lot of sources.

12. Patterns of electric light on Earth seen from space at night show the digital divide between north and south. Patterns of electric light on Earth seen from space at night show the digital divide between north and south. (UNEP report) Mapping Science http://www.nasa.gov/mission_pages/NPP/news/earth-at-night.html

13. NIDCD Data Set Portable Digital File format =.pdf

14. NIDCD Data Set Scalable Vector Graphics format =.svg

15. NIDCD Data Set

32. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH The NIAID question  What is NIAID’s impact on society? How has NIAID’s work influenced standards of care?  The Standard of Care  Prevention and Control of Influenza with Vaccines  Recommendations of the Advisory Committee on Immunization  Practices (ACIP), 2010  Morbidity and Mortality Weekly Report www.cdc.gov/mmwr

33. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH How to Assess Impact how many intramural NIAID authors are cited in the references at the end of this standard or care? how many papers in the reference listing were funded with NIAID grant funds?

34. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Logistics 552 citations in Scopus 524 export Lose 7 Lose 69--not indexed in Medline Scopus to EndNote to Medline to WOS to EndNote Dynamics of dedupping –Select criteria carefully –Rerun with different criteria

35. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH NIAID affiliation Intramural Extramural Address field Grants UNIQUE 11 59 10 54 67

36. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Conclusion Out of the 448 references* cited in the document Prevention and Control of Influenza with Vaccines Recommendations of the Advisory Committee on Immunization, NIAID is responsible for or affiliated with 67 or 15% * the document has 552 references but only 448 made it through the system without getting lost

37. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Who else is citing the NIAID papers?

38. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH Can we predict future science with publication data? Reliable metrics Clean data Establish linkages –funding –papers –new science Evaluate trends

39. DIVISION OF LIBRARY SERVICES | OFFICE OF RESEARCH SERVICES | NATIONAL INSTITUTES OF HEALTH For more information Terrie Wheeler National Institutes of Health Library terrie.wheeler@nih.gov 301.496.1157