CRIStin, reporting and rewarding research Eli Heldaas Seland and Monica Roos, 12 May 2016

Outline: The Norwegian system for weighted funding Publication points as a principle for dividing funding between institutions NSD – register of publication channels Levels 1 and 2 Evaluation and recent changes to the system Use and analysis of data Cristin Cristin and the library

The Norwegian Publication Indicator System from 2004, for: documenting Norwegian academic publishing measuring publication activity allocating research funding according to performance Distributes approx. 2 % of total funds for university and university college sector Evaluation in 2014 resulted in some changes

The original purpose of the indicator …to create a performance-based funding model for research and to encourage more research activity at universities and university colleges. The documentation system must also be flexible so that it can be used for other reporting needs and in individual and institutional publications lists, applications, project reports and evaluations, annual reports and information for the public at large http://www.uhr.no/documents/Rapport_fra_UHR_prosjektet_4_11_engCJS_endelig_versjon_av_hele_oversettelsen.pdf, p.3.

Register of publication channels Norwegian Register for Scientific Journals, Series and Publishers: authorized register of approved publication channels, divided into levels 1 and 2. A publication in a publication channel on this list, which fills specific criteria, is eligible to a certain amount of publication points – which in turn generate funding for the author’s institution (and, in many cases prestige for the authors)

What defines an academic publication? An academic publication is defined according to four criteria; all four of these must be satisfied. The publication must: present new insight; be presented in a form that allows the research findings to be verified and/or used in new research activity; be written in a language and have a distribution that make the publication accessible to most interested researchers; appear in a publication channel (journal, series, book publisher, website) that has routines for external peer review. http://www.uhr.no/documents/Rapport_fra_UHR_prosjektet_4_11_engCJS_endelig_versjon_av_hele_oversettelsen.pdf, p.12.

Levels 1 and 2 Level 1 scientific or academic publication channels (national and international) Level 2 The most important / highly regarded publication channels, maximum 20% of a subject area’s total scientific publications National academic councils/committees maintain the two levels. Anyone can nominate publication channels.

Weighting of publication types for the two levels of publication channels Monograph 5 8 Article in periodical or series 1 3 Article in anthology 0.7

Fractionalization Originally: A publication with 1 author is worth 1 point. A publication with n authors is worth 1/n point for each author, but the value must be at least 1/10 point. (Fractions are not used when there are more than 10 authors.) An institution is granted the sum of the point values from those authors who credit the institution.

2014 evaluation: Has the Publication indicator stimulated more research and research of higher quality? Publication volume/activity has increased, particularly with researchers and institutions that published little before Other parameters remain stable citation impact national and international cooperation little sign of salami publishing little change in publication types moderate shift from Norwegian to English language

Problems identified: The increase in output has not been accompanied by an increase in impact Lack of neutrality across disciplines: unintentional redistribution of funds between main areas Legitimacy in nomination processes Implementing incentives in institutions – exaggarated use at individual level

Solutions I Stimulate cooperation, nationally and internationally, by introducing the factor 1.3 for any publication with an international co-author

Solutions II: New way of calculating points that helps towards neutralizing across disciplines: √1/n

Solutions III New terminology Points «earned» are more difficult to connect to individual authors Discussions about local use is encouraged

What do we do? Create reports for strategic use for the institution, and for faculties, on request Prepare data for evaluations by the research council and other national committees and boards that ask for them Train and advice researchers in and about the system

CRIStin is a national research information system for: * Higher education sector * Research Institutions * Health sector CRIStin is also an organisation responsible for lisences, the CRIS-system and they are the national Open Access agency.

One system composed by several parts: One part to registrate activities (publications, conferences, music performances and so on) One part for projects One admin part for admin users Several other parts which are not much used (research leave)

Serves two purposes for researchers and research: As a «CV» of activities (researchers, but also admin staff) Publications, seminars, conferences, music/theatre performances, workshops…)

2. It is the tool used to do the actual «counting» of research publications: When level publications (level 1 and 2) are registrated, they end up in a «list» sent once a year to the government. - The library is responsible for administration, and the publication board for the professional evaluations.

Scientific material (articles, books and chapters) are the «countable» publications in the Norwegian weighted funding system. They must be peer reviewed They must be author affiliated They must be written for, and distributed to peers.

Cristin and the Library I The scientific material is checked and «approved» or «not approved» by the library Based on the formal «rules» provided Based on other institutions decisions * If a publication has several Norwegian authors, all «home» institutions have to agree on an approval for the publication to actually be reported. If a publication needs professional evaluation the publication board is doing this.

Cristin and the Library II The library is the administrator of the system (Cristin manager) We are training staff in registration and other uses of the system We are checking the list of scientific publications and sending the report We are in close cooperation with the research admin and the researchers We are part of a national network, influencing how the cristin system should evolve

Cristin and the Library III We are working with the communication department which is using cristin to harvest information about scientific publications and projects on to the HiB internal and external website We are responsible for any national changes in the reporting instructions, making sure they are implemented to our local (HiB) routines After the report is sent, the library starts working on the metrics, analysing this years numbers

Benefits for the library We do metrics We get a «say so» in digital work at HiB as Cristin is a master system on research in Norway The leaders of the institutions must come to the library to ask for results Researchers, teachers and others who are using the system must come to us We know everything there is to know about scientific publication in Norway (and Europe?) We gain knowledge on other related areas We are involved in strategical research planning

Cristin 2.0 The CRIStin system is old, built on old technology and serving old needs in the sector. A Cristin 2.0 is on its way. They are moving the system piece by piece over to new technology. - They are aiming at a finish in 2018 – having a more flexible system

Links: A Bibliometric Model for Performance-based Budgeting of Research Institutions http://www.uhr.no/documents/Evaluation_of_the_Norwegian_Publication_Indicator___English_Summary.pdf http://www.phdontrack.net/evaluation-and-ranking/weighted-funding-norway/ https://dbh.nsd.uib.no/publiseringskanaler/Forside.action?request_locale=en Cristin