A month in the life of a university bibliometrician Dr Ian Rowlands University of Leicester, UK

THE UNIVERSITY OF LEICESTER MY ROLE A TYPICAL MONTH Part 1: CONTEXT

University of Leicester Colleges Arts, Humanities & Law Engineering & Science Medicine & Life Sciences Social Sciences People 21,800 students (7,700 distance) 1,300 academic staff Research income Research grants and contracts £ 55 million (2013) REF-related QR income £ 24 million (2013) Famous for DNA fingerprinting Space science Richard III

David Wilson Library

Library Research Services Research analytics Research data management Researcher training and development Institutional repository Gold open access funds Interface with University systems Briefings on information issues PhD training (library tools) Project support Bibliometrics support Internal consultancy Management reports

My diary March 2014 Staff training and development Briefing on journal impact factors Briefing on researcher identifiers Live webinar on h-index Develop `citation tips’ leaflet Reports Background report on QS World University Rankings for Vice-Chancellor Analysis of collaboration between universities in the English Midlands Internal review of Department of Chemistry Major enquiries Can we use citation data to help thin out the print journal collection? How can citation data be used in a systematic health review? Quick enquiries Handling Spanish surnames Finding someone’s h-index Routine maintenance Quarterly update of InCites address database

Citations 66.9 Citations 56.3

RESEARCH EXCELLENCE FRAMEWORK USE OF CITATION DATA IN REF2014 Part 2: REF2014

REF2014 REF2014 is a UK-wide framework for assessing research in all disciplines. Its purpose is: to inform research funding allocations (approximately £2 billion per year) provide accountability for public funding of research and demonstrate its benefits to provide benchmarks

Assessment framework Overall quality Outputs Maximum of 4 outputs per researcher Impact Impact template and case studies Environment Environment data and template 65% 20% 15%

Assessment criteria The criteria for assessing the quality of outputs are originality, significance and rigour* Four star Quality that is world-leading in terms of originality, significance and rigour Three star Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence Two star Quality that is recognised internationally in terms of originality, significance and rigour One star Quality that is recognised nationally in terms of originality, significance and rigour Unclassified Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment

Highest possible grade … or highest possible volume?

Assessment criteria The criteria for assessing the quality of outputs are originality, significance and rigour* Four star Quality that is world-leading in terms of originality, significance and rigour Three star Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence Two star Quality that is recognised internationally in terms of originality, significance and rigour One star Quality that is recognised nationally in terms of originality, significance and rigour Unclassified Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment NOT FUNDED

REF2014 and use of citation data The following subpanels will make use of citation data: - Main Panel A: Subpanels 1-6 [life sciences] - Main Panel B: Subpanels 7-11 [physical sciences] - Main Panel C: Subpanel 18 [economics] REF2014 will use only use Scopus citation data None of the sub-panels will use journal impact factors, journal rankings, or other forms of bibliometric analysis

How we used citation data Each paper was graded by two assessors, at least one external to the University We counted Scopus citations and used published REF2014 calibration tables to locate papers in the top 1%, 5%, 10% or 25% of world impact We repeated the exercise using Web of Science InCites, where we have continuous citation percentiles and other metrics such as journal impact factors

How we used citation data Citation data showed a very good fit with human judgments, so were a helpful source of additional information The data were particularly useful in guiding decisions at the critical 3*/2* boundary The data helped us to model the whole submission and (hopefully) optimise the trade-off between grade and volume

?