Dr Amberyn Thomas Manager, Scholarly Publications University of Queensland, Australia

* Background * The research landscape, with a particular focus on evaluation * The UQ Library structure and services offered * Who we are engaging with and examples of what we do * Training and suggestions for best-practice

* Research-intensive 6 faculties 33 schools 9 research institutes * 47 sites Hospitals, Islands & Mine * 48,800 students Postgraduates – >12,580 * Staff (FTE) Total >6,890 Academic > 2,880

One of the top universities in Australia and in top 100 worldwide State based university National university Global university

* Many countries have research evaluation programs * e.g. UK RAE since 1992 (now REF) * Within Australia: * ERA 2010 & ERA 2012 & will run again in 2015 * Increasing prominence of university rankings, and new rankings emerging: “Universities are increasingly confronted by a plethora of ranking and classification initiatives” - European University Association * ‘Globalisation of research’ and relationship between ‘collaboration’ and ‘quality’ * New metrics – altmetrics and usage statistics * Shift to include broader ‘impact’ Impact ‘locally’ – within UQ there is keen focus on research quality & impact, & projects aimed at understanding collaboration

 UQeSpace (institutional repository)  Digitisation  Bibliometrics  Scholarly Publishing, Open Access  Research publications reporting and evaluation  Research Data Management (DM plans, repository)  Copyright Deputy Vice Chancellor (Academic)

eSpace repository ResearcherID Web of Science Scopus Citation Counts & Publication Records Data entry Publication records Citation counts & Publication Records UQ Staff Data (HR) Government reporting, ERA, UQ data warehouse, Academic portfolio, UQreSEARCHers, etc My UQ eSpace Altmetrics Scopus Custom Dataset Benchmarking InCites Author Profile

* Involved in Excellence in Research for Australia ERA 2010 & 2012 * Provide briefing notes and information about metrics-related data for e.g. ODVC(R) * University ranking information * Nature and Science publishing activity * Benchmarking data (bibliometrics) * Produce reports on collaborative publication activity for: * International Office, Research Office, Senior Executive * Provide reports and metrics advice for School Reviews & major grant applications * Work closely with Research Information Service Librarians who work with individual academics, research groups and Schools in the provision of metrics services and advice

* Individual researchers and research teams * Heads of School, Faculty, Institute Directors * Research managers and administration staff * Research Committees * Senior exec: VC (President) Office, Deputy Vice-Chancellor (Research), International office * Commercialisation Unit – UNIQUEST at UQ

* Advise academics on what tools are available and how useful they are for tracking research outputs & their citations * E.g. Web of Science, InCites, Scopus, ORCiD, Altmetrics * Help academics identify and find relevant metrics, e.g. h-index * Provide context for metrics, including benchmarking and expected citation rates for different discipline areas where appropriate * Advise on how to put the metrics into broader context * Review what academics write using metrics information and provide feedback on accuracy, relevance etc.

qual·i·ty/ ˈ kwäl ə tē/ Noun the standard of something as measured against other things of a similar kind the degree of excellence of something

Metrics provide supporting evidence of claims individuals and research teams make Metrics provide supporting evidence of claims individuals and research teams make e.g. Track record of individuals/teams * Contribution to the discipline and/or significance of the contribution * Evidence of international/national profile * Evidence of capacity to collaborate effectively

* Collaboration reports – country & institution * University rankings information * Key Performance Indicator information * Annual report statistics * School Review support

* What industries do UQ authors co-publish with? * What industries do academics in other Australian-based institutions co-author with? * What industries are acknowledged as a ‘funder’ of UQ publications? RankInstitutionWeb of Science Documents 1BAOSTEEL CO1 1MONASH UNIV1 1SHANGHAI JIAO TONG UNIV1 1UNIV TECHNOL SYDNEY1 5BAOSTEEL STAINESS STEEL CO IRONMAKING PLANT 1 5BHP BILLITON CHINA1 5BHP BILLITON NEWCASTLE TECHNOL CTR1

* Ensure adequate training opportunities and support for Librarians to ‘learn while doing’ * In the beginning: seminars & hands-on workshops for Librarians by specialist * Conducted scenario-based training * Librarians worked in groups of 2 or 3 over a 6 week period then the groups presented their results to each other in a ½ day workshop * Scenario examples: * I would like advice on how to benchmark my research performance against others in my field within the Australian context, as well as internationally. I am in the sciences with good WoS and Scopus coverage. * What is my h-index? I am in a discipline with excellent Web of Science and Scopus coverage – I am in Biological Sciences. Why is the Scopus value different to the WoS h-index? How does Google Scholar compare? * I would like to demonstrate in my research grant application that my research has found widespread application beyond my discipline. How can I do this? I am in a discipline with good WoS and Scopus coverage.

* Worked very closely with Librarians when they first started working with academics in this area * Attended meetings with them * Provided outlines of suggestions * Proof-read written material * Developed resources for Librarians and made these available on the intranet * Self-directed learning tasks * InCites training activity * Provided work rotations for Librarians in SPADS unit * Conducted regular update sessions And training is ongoing…..

The Library does not offer a standard bibliometric profiling service, but will work with individual researchers to provide targeted, contextualised information, and detailed advice about what tools and methods are available for tracking research outputs and their impact. This approach reflects the significant disciplinary differences in the application of metrics solutions for research evaluation. Further, the service aims to provide relevant information, which will come about through understanding the context in which the metrics are being used. The Research Metrics Service will support the University’s mission and enhance world-class scholarship. In alignment with this mission, the Library Research Metrics Service will work with School, Institute and Centre Heads, as well as managers of smaller research groups and centres to provide relevant and useful metrics information. Resources permitting, the Library will be proactive in supporting UQ’s participation in national initiatives in the area of research evaluation, such as the Excellence in Research for Australia, which align with UQ’s strategic goals.

* Librarians are not ‘bibliometricians’ * For individual researchers and small research teams, Librarians deliver a service based on provision of advice and practical assistance * Requires relevant resources & sophisticated tools * Demands ongoing training and being kept up-to- date with scholarly publishing trends

Don’t reduce individual performance to a single number Don’t use IFs as measures of quality Don’t apply (hidden) “bibliometric filters” for selection Don’t apply arbitrary weights to co-authorship Don’t rank scientists according to 1 indicator Don’t merge incommensurable measures Don’t use flawed statistics Don’t blindly trust one-hit wonders Don’t compare apples and oranges Don’t allow deadlines and workload to compel you to drop good practices Wolfgang Glanzel (Leuven), Paul Wouters (Leiden): ISSI 2013 Plenary Session

Also individual-level bibliometrics is statistics Analyse collaboration profiles of researchers Always combine quantitative and qualitative methods Use citation context analysis Analyse subject profiles Make an explicit choice for oeuvre or time-window analysis Combine bibliometrics with career analysis Clean bibliographic data carefully and use external sources Even some “don’ts” are not taboo if properly applied Help users to interpret and apply your results Wolfgang Glanzel (Leuven), Paul Wouters (Leiden): ISSI 2013 Plenary Session

Thank-you!