1. Research Assessment Exercise 2005 (RAE2005) University of Helsinki Arto Mustajoki Based on the material of Vice-Rector Marja Makarow

2. 2 Assessment of university research University of Helsinki (UH) University Senate’s decision RAE of UH every 6th year 1st time in 1999 (first in Finland) 2nd time in 2005 University of Tampere 2004 University of Jyväskylä 2005 Concept different from UK RAE Carried out by UK Council, every 8th year Peer review, no site visits Important financial consequences

3. 3 Why a RAE at the UH? Current challenges for universities in Finland Profiling Strategic spearheads of research International competitiveness Productivity progam Financial constraints How to best meet the challenges Knowledge of strengths, weaknesses and potential External evaluation yields solid objective data on quality of research recommendations for the future

4. 4 What was evaluated? Quality of research of departments Grade 1-7 (7 is best) Verbal arguments Concepts of institutes, research networks and stations Only verbal arguments No grades Interaction with society Only verbal arguments No grades

5. 5 What was evaluated? 1Quality of research Quality of research compared to that of SIMILAR EUROPEAN INSTITUTIONS Expressed with grades from 1 to 7 Substantiated with verbal comments NO comparison to results of UH REA 1999 NO comparison between UH departments

6. 6 Target of assessment All research performed in UH during 1999-2004 Units of assessment Faculty departments: 70 in 11 faculties Research institutes independent of faculties: 5 Individual researchers or research groups were NOT evaluated

7. 7 What was evaluated? 2Concepts Verbal evaluation only of Research (field) stations: 5 Research networks: 5 Independent institutes: 5 Verbal evaluation of quality of research: Medical research programmes: 6

8. 8 What was evaluated? 3The third task – interaction with society Pilot project Legal obligation of universities since Aug 2005 Documentation of accomplishments not systematic For example Expert tasks, popularization of science, text books, clinical and commercial application of research data Units of assessment chose what to report Evaluators commented verbally The data serves to Clarify how new knowledge has been tranferred to be used in society Develop and document activities

9. 9 How was the assessment carried out? Peer review Documents on acitivities of 94 units of assessment covering 1999-2004 via Evaluation Office to evaluators Desk work at home Preliminary drafting of Evaluation Report Each panel one working week in Helsinki General info on Finnish science policy, university system, UH Site visits to premises, infrastructure Interviews, researchers, PhD students, post-docs Writing of Evaluation Report before leaving Finalizing of the Report by Panel Chair Editing the Report in Evaluation Office

10. 10 Who were the evaluators? 148 mostly international scientific experts, in 21 panels From 21 countries 83% from Europe 9% from Finland 12% were there in 1999 18% from LERU universities 30% of panelists and chairs were women Panelists were chosen from suggestions obtained from Scientific Council of UH Chairs of national Research Councils Rectors of LERU universities Intl top scientists

11. 11 Research active staff (RAS) in 1999-2004 Researchers, post-docs and PhD students 4,000 annual work years Results of altogether 24,000 researchers’ work years was evaluated Number of RAS per unit of assessment: 3 - 640 Humanities and social sciences: 3-76 Natural science: 14-154 Medicine 24-640 Agriculture, Forestry, Bioscience, Pharmacy 7-340

12. 12 Publications in 1999-2004 Only publications in the official data base of UH were encluded in the assessment Altogether 60,000 publications 21,000 peer reviewed publications 22,000 other publications 2,400 monographies 2,000 PhD theses 10,000 popularized publications 600 text books

13. 13 Competitive funding fetched by the researchers in 1999-2004 (kiloEUR) Research Councils (Academy of Finland)220.000 Ntl Technology Agency (Tekes)61.000 Ministries127.000 Ntl foundations37.000 Intl foundations27.000 EU FWPs49.000 TOTAL521 M€

14. 14 Results 1Grades of the quality of research Average 1999 4,66 Average 2005 5,8

15. 15 Results 3Summary From the 75 units 66 were evaluated also in 1999. From them 29 (44%) improved their grade 31 (47%) got the same grade 6 (9%) got a lower grade 20 units (27%) got the best grade 7 6 units improved from 4 to 6 1 unit improved from 3 to 6

16. 16 Criteria of the grades 7: >50% of submitted works are at high intl level and all others are at good intl level 6: >33% at high intl level and many others at good intl level 5: >50% at least at good intl level and others at fair intl level 4: >33% at good intl level and many others at fair intl level 3: >50% at least at fair intl level 2: >50% at fair intl level 1: none at fair intl level

17. 17 Quality of research increased - why? Performance of individual researchers and teams Structural development at university level Concentration of activities to 4 campuses Fusion of departments (115>75) Evaluation culture adopted Recommendations of RAE1999 implemented Background Sufficient national resources for research Intelligent national science policy

18. 18 Principles of financial consequences of results University Senate’s decision before publication of results The best units and faculties are rewarded The resources will come from the university’s private funds, not state budget No unit or faculty looses resources due to poor performance

19. 19 Rewards to the best units of assessment Units of assessment 1.600 €/RAS/year will be awardes in 2007-2012 to units which obtained grade 7 1.600 €/RAS/year in 2007-2009 to units which improved to grade 6 from grade 3 or 4 27 units to be rewarded annually with 30.000 - 288.000 €

20. 20 Rewards to the beast faculties Faculties, rewarded during 2007-2009 6 faculties with the best average grade Amount of reward relative to number of RAS after deduction of the RAS of units to be awarded directly 6 faculties to be rewarded annually with 32.000 – 309.000 €

21. 21 Total investment to quality research in 2007- 2012 12 M€ to units of assessment 3 M€ to faculties Total investment 15 M€

22. 22 Evaluators’ comments to leadership Structures which best support quality research in universities Independent institutes Research programmes Collaboration and strategic alliences Infrastructure and its sharing Proactive recruitment of researchers Funding of research Allocation of time for research Researchers’ careers Leadership

23. 23 Governance of RAE2005 Director of RAE2005 Vice-Rector for Research Prof. Marja Makarow Steering committee (Chair prof. M. Makarow) Prof. A. Mustajoki Director U. Mansikkamäki Mr. H. Kallasvaara Panels and panelists: Steering committee >Reasearch Council of UH ToR for evaluators & Guidance for units of assessment: Evaluation Office (K. Haila & R. Holm) > Steering Committee > Reasearch Council of UH Principles of financial consequences Research Council of UH Decision by University Senate

24. 24 Publication of results Duration of RAE procedure from May 2004 to March 2006 Site visits of panels in Helsinki May-June 2005: panels 1-4 September-November 2005: panels 5-21 Publication of results on the web on March 1, 2006 Summary Report (also available in printed fomat) Individual Evaluation Reports www.helsinki.fi/research2005