Ekkehard Nuissl von Rein Quality Assurance by External Evaluation of Leibniz Institutes Strasbourg, 15th November 2005

1.The Leibniz Association 2.External Evaluation Procedure of the Leibniz Association 3.Results of the External Evaluation

Applied Research Fraunhofer Society Basic Research Max Planck Society Helmholtz Association Leibniz Association Problem-oriented Research German Publicly Funded Non-University Research System 1. The Leibniz Association

The 84 research institutes and scientific service facilities of the Leibniz Association are divided into sections: A: Humanities and Education Research (14 institutes) B: Economics, Social Sciences and Regional Infrastructure Research (17) C: Life Sciences (21) D: Mathematics, Natural Sciences and Engineering (23) E: Environmental Sciences (9) 1. The Leibniz Association

Leibniz Institutes address specific scientific problems, which are socially relevant e.g. – Diabetes Research – Climate Research offer consulting services for different reference groups, e.g. – Economics – Adult Education offer information or infrastructure services for research, e.g. – Primate Centre – Information Centre for Social Sciences 1. The Leibniz Association

Location of the 84 Leibniz Institutes

Funding Non-profit research organisation Institutes are co-financed by the federal and the 16 state governments Budget of about one billion €, thereof 20 % from third-party funding About 12,000 employees, about 7,000 academics 1. The Leibniz Association

Section A Section B Section C Section D Section E Administrative Board Leibniz Senate Organization of the Leibniz Association Administrative Office Evaluation Office President

Quality Assurance System Scientific excellence and service quality is ensured by three elements: Internal quality management by the institutes Accompanying consulting and evaluation by the Advisory Boards of the institutes External Evaluation by the Leibniz Senate 1. The Leibniz Association

Accountability: Assessment whether the institute meets the requirements of further public funding formulated as evaluation criteria Improvement: Recommendation for the institute‘s development Objectives of the external evaluation procedure are 2. External Evaluation Procedure

Evaluation criteria 1. Scientific Quality of research and service: determining programmatic focus areas and future prospects; theoretical and methodological foundation of research; work efficiency and effectiveness; raising third party funds 2. External Evaluation Procedure

Evaluation criteria 2. National and international significance: International standing; networking; promoting young scientists; political, economic and social relevance 2. External Evaluation Procedure

Evaluation criteria 3. Structure and organization: Efficiency of management and organization, human resource development, internal quality management 2. External Evaluation Procedure

Methods Analysis of available documents (budgets etc.) Peer review by an expert panel Statistical methods, bibliometric data (if available) Visit of the institute‘s departments and interviews with the institute‘s staff 2. External Evaluation Procedure

4 Steps of the external Evaluation Procedure: (1) Institutes report on their work (3) Leibniz Senate discusses and passes statements (4) Political Committee (BLK) decides about further funding (2) International expert panels evaluate the quality of work

The evaluation procedure is based on principles: Regularity Independence Transparency Acceptance 2. External Evaluation Procedure

Crucial Points of the Evaluation Procedure Appropriateness of criteria Selection of experts Acceptance of recommendations 2. External Evaluation Procedure

3. Results of the External Evaluation  41 institutes have been evaluated since 2002  The Leibniz Senate has issued 24 statements: - 19 institutes will be supported for the next 7 years - 4 will be re-evaluated in 3 years - 1 institute will be partly merged

Statistics Costs per evaluation 55,000 € on average Number of experts10 to 12 Experts‘ nationalities A, AUS, CH, DK, F, GB, GER (75 %), NL, S, USA Professional work areas of experts university research (65 %), non-university research (15 %), economy/industry (8 %), associations, public authorities Evaluations per year12 to 14 Duration of a complete evaluation about one year

Quality Assurance by external evaluation? Observations on institutes’ performance development: In general institutes have - focused their research and service profile - increased scientific output, e.g. papers in reviewed journals - strengthened international co-operations - implemented internal quality management - emphasized promotion of young scientists.

Detailed information can be found on our homepage e.g.:  The Senate’s statements and evaluation reports  Questionnaires, tables and information on the evaluation procedures as well as the Senate Evaluation Committee