1. 1 Using metrics to your advantage Fei Yu and Martin Cvelbar

2. 2 Metrics Things that can be counted number of publications citation counts Also: Journal quality measures – e.g. Journal impact factor H-index Collaboration metrics Metrics: must always be considered in context – significant disciplinary differences, and differences over time. Using metrics to your advantage

3. 3 Why use publication metrics? 1.Metrics provide supporting evidence of claims individuals/research groups make Track record of individuals, groups of researchers Contribution to the discipline and/or significance of the contribution Evidence of international profile Evidence of capacity to collaborate effectively 2.The current environment … research evaluation now part of the landscape (ERA) – grant application reviewers, potential employers etc. may expect to see metrics Using metrics to your advantage

4. 4 Research Quality Determined by peers peer review AND/OR Inferred by “numbers” – bibliometrics indicators Citation counts etc. Using metrics to your advantage

5. 5 qual·i·ty/ˈkwälətē/ Noun the standard of something as measured against other things of a similar kind the degree of excellence of something Given the numbers some meaning! Using metrics to your advantage

6. 6 For papers published in journals within the ESI field of Mathematics in 2002 the average citation rate in WoS is 4.91 In Molecular Biology and Genetics it is 37.24!

7. 7 Citation counts As a research quality or performance measure the assumption is: high citations = high quality BUT “…not only the content of scientific work, but also other, in part non- scientific, factors play a role in citing behaviour. Citations can therefore be viewed as a complex, multidimensional…phenomenon” Lutz Bornmann, Hans-Dieter Daniel, (2008) "What do citation counts measure? A review of studies on citing behavior", Journal of Documentation, Vol. 64 Iss: 1, pp.45 - 80 Using metrics to your advantage

8. 8 Journal Impact Factor ….. Flawed for a number of reasons…but it may be a useful tool to profile researcher output … … Using metrics to your advantage

9. 9 Example “My research output is of very high quality, with publications consistently in good quality journals … “ What % of your journal articles are in the top 10% by Impact Factor for the Journal Citation Reports categories you publish in? Using metrics to your advantage

10. 10 Metrics: the standard 5 1.Number of publications 2.Career citation count 3.Citations per paper 4.% cited (or % uncited) 5.h index These numbers tell us something … but there’s a lot missing … … Using metrics to your advantage

11. 11 How to use metrics to paint a complete picture Using metrics to your advantage

12. 12 Provide more detail … … 1.Number of publications Profile by document type, subject area, year, journal etc. 2.Number of citations (career citation count) Provide an indication of coverage by citation data provider Are all outputs included or just articles and reviews? 3.Citations per paper I have 35 refereed journal articles, of which 33 are indexed by Web of Science. These articles have received 230 citations, giving an average citation per (indexed) paper of 7 (source: WoS, 05/01/15). 4.Percentage not cited Of my 33 indexed journal articles, 3 articles have not been cited by others (9% not cited), and these were all published in 2014 (WoS, 05/01/15). Using metrics to your advantage

13. 13 And some additional metrics… … Citations for an individual paper or sub-set o publications: supporting evidence for outstanding contribution in a particular area. Citations per paper by year of publiation and by journal/subject area/FOR code Can then benchmark E.g. Incites, or use Thomason Reuters Essential Science Indicators for benchmarking Or create your own benchmarks using refine reature within WoS Using metrics to your advantage

14. 14 5. h-index … …. Productivity + Impact = Influence Using metrics to your advantage

15. 15 Does it mean anything? Use the h-index measure with care. Citation patters vary across disciplines. E.g. h-indices in Medicine are much higher than in Mathematics for example. Researchers in different disciplines can not be compared using the h-index. Even within the same discipline the h-index should not be used alone as a measure of research quality. Researchers in the same discipline at different stages of their careers can not be compared using the h-index. Using metrics to your advantage

16. 16 Consider the following 3 researchers who have all published 10 papers and all have an h-index of 5 Using metrics to your advantage Researcher 1 cites per paper Researcher 2 cites per paper Researcher 3 cites per paper Paper 11005015 Paper 2905015 Paper 3704514 Paper 4504514 Paper 535512 Paper 6444 Paper 7024 Paper 8004 Paper 9004 Paper 10004 Average cites per paper35.320.19.0

17. 17 Put it in context … … My h-index based on these indexed papers is 10. I have 4 papers (A, B, C, D) with more than 20 citations and 1 paper (E) with 29 citations (source: WoS, 05/01/15). I also have an additional 3 papers not indexed by WoS, with 29 citations based on Scopus data (05/01/15). Using metrics to your advantage

18. 18 More examples … … “My work is multi-disciplinary, spanning biochemistry, biophysics and oncology …” Analyse your WoS articles by WoS subject category to see if this is evidenced in your research output. “34% of my journal articles are in the WoS Subject Area of Biochemistry and Molecular Biology, with 29% in Biophysics and 16% in Oncology (WoS Subject Areas, 05/01/15)”. Using metrics to your advantage

19. 19 More examples … “My research has found (widespread) application beyond my discipline … “ Look at the articles which cite your work in WoS and analyse them by WoS Subject Area. “Whilst 65% of my articles are published in Physiology (WoS Subject Area), 55% of my citations are from journals with a Subject Area of Cell Biology (source: WoS, 05/01/15).” Using metrics to your advantage

20. 20 For Early Career Researchers… with a short track record Look in detail at the quality of the citations your work is receiving What are the impact factors of the journals in which your work is cited What fields are the journals in? Who is citing your work? Who are they? Where are they from? Why are they citing your work? How quickly is your work being cited? Using metrics to your advantage

21. 21 Beyond bibliometrics….. Using metrics to your advantage Altmetrics is the creation and study of new metrics based on the Social Web for analyzing and informing scholarship. From http://altmetrics.org/,aofesto/http://altmetrics.org/,aofesto/

22. 22 Using metrics to your advantage

23. 23 Consider other measures of research impact … … for example: Mentions in legislation, government policy etc. Academic library holdings Look at these “56 indicators of impact”. http://cas-csid.cas.unt.edu/?p=4475 http://cas-csid.cas.unt.edu/?p=4475 See also: Using metrics to your advantage

24. 24 Some questions to think about: Have you viewed your download statistics or abstract views in eSpace? [countries & dates viewed] How many of your books are held in libraries worldwide? [library can provide data] Have you been invited to speak at conferences, and were the papers peer- reviewed? Has any of your work been commissioned? Has your research influenced government or education policy? Has there been any measurable social benefit from your research? Have you collaborated with others domestically or internationally? Have you received any awards? Using metrics to your advantage

25. 25 Some advice … … Use author disambiguation tools Google Scholar Citation profile ResearcherID Scopus authorID ORCiD (Open Researcher and Contributor ID) Using metrics to your advantage