Research evaluation: is it our business? Librarians in the brave new world of research evaluation Andria McGrath Senior Information Specialist, Research Support, Kings College London

What is research evaluation? External evaluation – in the UK the RAE now REF 3 major elements of the RAE: – publication outputs, – grant income awarded, – research students Vital importance – link to income Benchmarking – Rankings Internal evaluation

The players in institutions Research Office PVC Research Research group/dept heads Administrators Librarians – publications expertise Institutional repository managers 3

Evolving interest REF events at Kings st event – Beyond the RAE – representatives from 75 universities – 1/3 librarians Survey – – biggest challenges – verifying publication lists – Biggest concerns for REF – bibliometrics – data sources, subject differences, skilling up to cope 4

Events Nov 08 and June 09 Focus on bibliometrics – pilots Pre-event survey before Nov event: – Only about half of institutions had a centralized publications data collection system – One or two mentioned Symplectic – One mentioned something called Pure 5

Data collection systems Some – only concerned with collecting RAE pubs Others – aimed at collecting all publication info Wider uses – profile pages / internal evaluation Some creating internal home grown systems – Separate systems for 3 RAE data elements – Integrated system, taking feeds from other systems (eg Research Gateway at Kings)

REF focus changed Bibliometrics element toned down Impact (4 th Kings REF event in June 2010) Not publication impact – practical impacts of research Research leaders had seen the light on – data collection systems – bibliometrics 7

CRIS Current Research Information Systems Already common in mainland Europe 8

CERIF Common European Research Information Format A standard – uses XML Objects or entities with attributes eg project, person, organizational unit; Relationships – 'linking relations Rich semantics – roles and time 9

R4R project - Ready for REF JISC project – Kings and Southampton CERIF4REF XML schema created Plugins for Eprints, Dspace and Fedora Transfer data to REF collection system Research admins, IR managers, systems staff and librarians involved 10

Librarians and CRIS systems St Andrews experience If research office led – get in on the ground floor Librarians have plenty of expertise to offer 11

Which librarians? Senior managers Subject/liaison librarians Research support librarians Instutional repository managers Information resources depts. – subs Enquiries staff 12

Strategic directions for libraries RLUK strategic themes 1 st – redefining the research library model New services / new roles for staff Value of libraries for research and researchers (RIN/RLUK report) RLUK – Reskilling for Research (forthcoming) Research support librarians 13

Some new areas for librarians associated with research evaluation CRIS systems & CERIF New ways of interacting with bibliographic / citation databases APIs – for publication or citation data Discovering other institutional systems eg grants DBs / student systems that feed into CRISs Bibliometrics principles and new products 14

New opportunities Showing our value – To administrators / research office – To academics and senior research leaders – To Graduate School Opportunities to advocate Open access 15

Bibliometrics – the new buzz word Awareness growing By librarians By research leaders in institutions By academics 16

Librarians – natural synergy Citation databases Citation reports Training offered – eg by Thomson Reuters Training points up limitations of basic measures Normalizing of measures crucial but difficult without specialised products 17

By research leaders A few initiates who understand the issues leading the way Percentiles tables in Essential Science Indicators used for evaluations H-index being used by some At least one institution has employed a bibliometrician 18

Academics Some beginning to take an interest Discovering: –ResearcherID –Citation reports in citation databases –Google Scholar Graduates – training opportunity – Making an impact 19

Holy grail Research information system Dashboard for research managers – eg for grant information Captures data with minimum effort Integrated with bibliometrics For internal & external purposes – evaluation and promotional 20

Is it our business? 21