1 18.10.2015 Making Good Use of Research Evaluations Anneli Pauli, Vice President (Research)

Slides:



Advertisements
Similar presentations
Estonian Pilot Action on Technology Assessment Rene Tõnnisson Institute of Baltic Studies.
Advertisements

06/02/ Mechanisms to enhance synergy between ERA-NETs within and between research councils Eili Ervelä-Myréen
Information and Communication Technologies (ICT) in the Seventh Framework Programme Coordination actions ICT Calls Jan- March 2012.
Professor Dave Delpy Chief Executive of Engineering and Physical Sciences Research Council Research Councils UK Impact Champion Competition vs. Collaboration:
1 © ACADEMY OF FINLAND Synchronized Calls Concept and Funding Stakeholders’ perspectives Marja Makarow Vice-President Brussels 15 January, 2014.
MIRA - WP 2 Observatory of Euro-Med S&T cooperation White Paper Coord. IRD (France) CNRS (Lebanon) MIRA Mediterranean Innovation and Research coordination.
Information and Communication Technologies (ICT) in the Seventh Framework Programme Support actions.
CYPRUS UNIVERSITY OF TECHNOLOGY Internal Evaluation Procedures at CUT Quality Assurance Seminar Organised by the Ministry of Education and Culture and.
Research Programme on Nutrition Food and Health ELVIRA ( ) “Novel high-quality and innovative research that in the long run helps consumers.
Planning and running a regional evaluation system: the Madrid experience Beatriz Presmanes Coordinadora de Programas.
EU’s 7th framework program ( ) Cooperation Ideas (=ERC) People Capacities European Research Council (ERC) ERC The result of strong pressure from.
Mindfuel at the heart of transport research Era-net Transport Flagship 2015 Call Procedures and Timeline Kick-off event and Infoday Brussels, May 13 th.
1 © ACADEMY OF FINLAND Academy of Finland 2014: Research knows no boundaries Tiina Jokela PhD, Senior Science Adviser, Strategic Research.
Association for the Education of Adults EAEA European AE Research – Look towards the future ERDI General Assembly, 2004.
Research Infrastructures The Research Infrastructures in FP7.
Regional Research Promotion Programme in the Western Balkans - RRPP Bojana Radovanovic RRPP LCU in Serbia “Making work pay in Western Balkan countries:
National Science Centre An Important Component of Research Funding in Poland RECFA visit to Poland, Krakow 2012 Andrzej Jajszczyk.
The evaluation of research units at HCERES
LIITO Renewing Business and Management Program Heli Kukko Technology Director The Finnish Funding Agency of Technology and Innovation LIIKE2.
SHOK Strategic Centers for Science, Technology and Innovation World-class breakthroughs in innovation cooperation.
Supporting entrpreneurs and innovators in Finland Timo Kekkonen Director, Confederation of Finnish Industries, EK.
Final evaluation of the Research Programme on Social Capital and Networks of Trust (SoCa) 2004 – 2007: What should the Academy of Finland learn.
STRATEGY OF LIFELONG LEARNING AT THE UNIVERSITY OF NIS
Enhancing quality of Finnish higher education – Impact of institutional audits Senior advisor Kirsi Hiltunen Finnish Higher Education Evaluation.
Advocacy and policy dialogue at national level on development education Seminar for NGDO Platforms of New Member States Budapest, 9-10th March 2006 Rilli.
Joint Programme Enhancing judicial reform in the Eastern Partnership countries Judicial component.
Adaptation knowledge needs and response under the UNFCCC process Adaptation Knowledge Day V Session 1: Knowledge Gaps Bonn, Germany 09 June 2014 Rojina.
1 © ACADEMY OF FINLAND Academy of Finland 2015: Research funding for basic science Tiina Jokela PhD, Senior Science Adviser, Strategic Research.
Towards a European network for digital preservation Ideas for a proposal Mariella Guercio, University of Urbino.
1 NEST New and emerging science and technology EUROPEAN COMMISSION - 6th Framework programme : Anticipating Scientific and Technological Needs.
The Evaluation of Publicly Funded Research Berlin, 26/27 September 2005 Evaluation for a changing research base Paul Hubbard Head of Research Policy, HEFCE,
Introduction to PROGRESS Community programme for Employment and Social Solidarity Finn Ola Jølstad Norwegian Ministry of Labour and Social Inclusion.
Strategic Centres for Science, Technology and Innovation DM Copyright © Tekes.
With the support of the Lifelong Learning Programme of the European Union European Lifelong Guidance Policy Network Coordinator:
WP8 – Innovation Support Kelly Vavasi General Secretariat for Research and Technology (GSRT) 1 st Innovation Dialogue Forum Becici, 8-9 November 2010.
Dr Ritva Dammert Director Brussels May 27, 2009 Evaluation of the Finnish Centres of Excellence Programmes
PP 4.1: IWRM Planning Framework. 2 Module Objective and Scope Participants acquire knowledge of the Principles of Good Basin Planning and can apply the.
Project Management Learning Program 7-18 May 2012, Mekong Institute, Khon Kaen, Thailand Writing Project Report Multi-Purpose Reporting.
The Experience of a Leader in Innovation. The Case of Finland Professor Reijo Vihko President and Director General The Academy of Finland.
DETERMINE Working document # 4 'Economic arguments for addressing social determinants of health inequalities' December 2009 Owen Metcalfe & Teresa Lavin.
María Amor Barros del Río Gender as content in research in Horizon 2020 GENDER AS CONTENT IN RESEARCH IN HORIZON 2020 CAPACITY BUILDING WORKSHOP FOR RESEARCHERS.
The RDI Governance System Vasileios Pitsinigkos Head of Managing Authority of Eastern Macedonia - Thrace Region.
ESF Member Organisation Forum Science in Society Relationships Inproving interaction with society – urge for strategy & action ESOF2012 session.
The partnership principle and the European Code of Conduct on Partnership.
ROMANIA MINISTRY OF EDUCATION, RESEARCH AND INNOVATION National Centre for Development of Vocational Education and Training Implementation Unit of Phare.
The ERA-NET TRANSCAN-2, in continuity with the preceding ERA-NET TRANSCAN, aims at linking translational cancer research funding programmes in 15 Member.
E u r o p e a n C o m m i s s i o nCommunity Research Global Change and Ecosystems EU environmental research : Part B Policy objectives  Lisbon strategy.
1 © ACADEMY OF FINLAND Academy of Finland 2012: Research knows no boundaries Tiina Kotti PhD, Programme Manager, Programme Unit.
Kathy Corbiere Service Delivery and Performance Commission
TEN-T Executive Agency and Project Management Anna LIVIERATOU-TOLL TEN-T Executive Agency Senior Programme and Policy Coordinator European Economic and.
The 7th Framework Programme for Research: Strategy of international cooperation activities Robert Burmanjer Head of Unit, “International Scientific Cooperation.
1 © ACADEMY OF FINLAND Academy of Finland 2013: Research knows no boundaries Tiina Jokela PhD, Programme Manager, Programme Unit.
SWAFS NCP Info Day Brussels, 2 February 2016 RTD B7 - Science with and for Society RTD-7.1 Gender Sector HORIZON 2020.
Pentti Pulkkinen Programme Manager Academy of Finland Research funding and administration in Finland
About the European Science Foundation 1. 2 ESF Member Organisations ESF is an independent association of 13 Member Organisations ● research funding organisations.
NIFU STEP studies in Innovation, Research and Education Peer review of impact? Options and challenges Liv Langfeldt RCN 14 April 2008.
Open data Program pp.kk.vvvv Osasto Finland Open data policy in Finland  Long history of Open Government  Freedom of Information Act in 1766.
The Finnish Innovation System and Comparisons with Scotland May 2007.
Research and Innovation Why does ERA Need to Flourish ERA priority 1: More effective National Research Systems Workshop for enlargement countries 10 March.
LIVING LAB OF GLOBAL CHANGE RESEARCH
Research Indicators for Open Science
LESSONS LEARNED FROM THE PRESENT GENERATION OF HIGHER EDUCATION PROGRAMMES IN EASTERN PARTNERSHIP COUNTRIES Klaus Haupt, Head of Tempus Unit Education,
Analytical Summary of Replies to the Questionnaire for Governments
Strengthening the foundations of ERA
Lifelong Learning policies and the Open Method of Cooperation
Higher Education and Research in Finland and the Vision of Future Reijo Aholainen Senior Ministerial Adviser Ministry of Education and Science Department.
Department of Teacher Education, University of Helsinki, Finland
Brussels, 15 September 2009.
Introduction to the training
Our vision Knowledge creates a sustainable world
Presentation transcript:

Making Good Use of Research Evaluations Anneli Pauli, Vice President (Research)

Academy of Finland Main tasks

Academy of Finland Organisation

Academy of Finland Evaluation activities

Benefits of using international peer review method …to the Finnish Science Scientific quality guaranteed Easier to achieve objectivity, equal treatment and transparency Less negative feedback from researchers (small country) May enhance international collaboration More international visibility

Evaluation activitiesImpacts Applications Programmes Disciplines and fields of research Research system  Excellent scientific quality is a prerequisite for scientific as well as societal relevance  Feedback received on the quality of research, scientific and societal relevance, interdisciplinarity, national and international cooperation, programme management; Impact on the development of the field, identification of future research needs  Measures for further development of the field, prioritisation, possibly resource allocation  Knowledge on the structure and dynamics of science and innovation system for its further development

Peer review of applications Scientific rating by the peers Ranking by the Research Councils % of peers are foreigners, depending on the call Applications in English

Research programmes From planning to implementation Follow-up and evaluation plans included in early stages of planning A programme is evaluated against its objectives and funding volume Main components of the evaluation added value of the programme, ‘programmatic nature’ scientific results and impact societal, economic, technological, cultural impacts implementation and management of the programme Utilisation of evaluation after-care plans of the issues raised by the programme evaluation identifying future research needs and directions

Research fields: impact of evaluations Thorough planning of the evaluation Timing of the evaluation Commitment of research community to develop the field (realistic expectations) other stakeholders (research councils, other funding organisations, ministries etc.) Follow-up within the process of management by results (Finnish Ministry of Education and Science and Academy of Finland) cooperation of researchers, research institutions, and other stakeholders

CASE: Evaluation of Biotechnology in Finland (1) 2002: Evaluation of Biotechnology in Finland. Impact of Public Research Funding and Strategies for the Future. Commissioned by the Ministry of Education and Science Coordinated by the Academy of Finland Involved multiple stakeholders: ministries, Academy of Finland, National Technology Agency of Finland Tekes, and Sitra, industry Carried out by an international expert panel, chaired by Professor Fotis C. Kafatos, then dir.gen. of EMBL 2003: A follow-up working group appointed by the Ministry of Education and Science (MoE) Made a proposal for future actions and implementation of the recommendations in : MoE working group “Development programme for research and training in biotechnology in ” Follows and evaluates the implementation of recommendations and measures based on the Biotech 2002 Evaluation and the work of the follow- up working group 2003

CASE: Evaluation of Biotechnology in Finland (2) 2005, MoE working group: Questionnaire to universities and their biocentres, research institutes and funders of biotechnology: Information about resources and impact of research Implementation of the recommendations made in the BIOTECH 2002 evaluation Which recommendations, how and when the organisation has implemented? Which recommendations and when will be implemented in the future? Which recommendations cannot be implemented and why? Description of any other development measures which have been carried out in Which of the recommendations or measures addressed to other organisations have been of little importance and/or have been implemented inadequately?

Research system Reviews on the State, Quality and Impact of Scientific Research (1997, 2000, 2003) Next : 1. Bibliometrics Analyses of Finnish science (e.g. fields, organizations, cooperation) Comparison of Finland to OECD countries’ performance 2. Impact evaluation International benchmarking of evaluation methods and indicators Academy’s four Research Councils: Impact assessments of funded research  External expert panel: Impact evaluation of the Academy 3. Foresight Joint project of the Academy of Finland and the National Technology Agency Tekes 10 expert panels, important issues in the future

Challenges: from planning to implementation What is the motivation for carrying out an evaluation Cost-effectiveness Selection of evaluators of crucial importance Stakeholders: multiple views, commitment needed Long-term planning vs. short-term needs

Conclusions Different needs, levels, implementation and impacts Key issues of concern to research communities and institutions / policy makers / managers / others define the type of the evaluation Clear aims and criteria stated for each evaluation Right timing Results in a user-friendly format Combining evaluation with strategic planning / forward-looking / foresight Impacts can only be achieved if the findings of the evaluation results are utilised.