1. The evaluation of research units at HCERES
May 11, 2015
The evaluation of research units at HCERES
Alain Cappy, coordinator for the S&T domain

2. Outline
1-HCERES présentation Status and Organization Founding Principles Missions and Objectives EU Recognition Annual Work IntegratedE 2-Evaluation of research units Principles Report Syntheses 3- Conclusion

3. HCERES: Status and Organisation
Status: Independent Administrative Authority
Organization
President, chairs the board
Board of the Council: 30 members
3 evaluation departments
Institutions
Research units
Programs and degrees
Observatory of Science and Technologies ( data base, bibliometric analysis, indicators…)
Support service

4. HCERES: Status and Organisation
Scientific organization
About 100 part-time scientific delegates
4,500 experts commited every year, including 20% outside France
Administrative organization
About 100 full-time administrative and technical staff
Budget in 2015: about 15 M€

5. Founding Principles Independence:
Status of independent administrative authority. HCERES to carry out its missions without coming under pressure from:
government authorities,
Evaluated entities ( research units, Universities, research bodies),
or any other stakeholders
HCERES guarantees compliance with the rules of ethics, in particular those concerning conflicts of interests.

6. Founding Principles
Competence of experts and transparency of procedures
Quality of evaluation rests on:
the expert’s skills and correct matching his/her profile with the entity evaluated,
a methodology and procedures guaranteeing equality of treatment,
drawing up by HCERES of a quality management system.
Transparency of evaluation lies on publication (website) of :
criteria for and methods of evaluation,
the list of scientific delegates, experts and their positions,
Summary of evaluation reports.

7. HCERES: Missions and Objectives
Evaluating:
HE & R institutions (universities, schools and research bodies), taking account of all their missions and activities.
Research activity laboratories and research units of these institutions
Education programs and degrees in the French higher education system (BMD)
Assisting evaluated entities in the continuous improvement of their practices
Providing clarification for decision-makers:
The State (financing, recognition of qualifications)
Research bodies (certification of laboratories)
Others (Local Authorities, Companies, Families)

8. What HCERES Do and Do Not
HCEES evaluates and does NOT decide
Decisions about accreditation and funding belong to Universities, Institutions, State,…
HCERES does NOT evaluate persons
But representatives of person evaluation bodies are included at different levels :
HCERES Council
Visiting committees

9. European Recognition
HCERES is member of the European Association for Quality Assurance in Higher Education (ENQA). ENQA promotes European co-operation in the field of quality assurance in higher education .
HCERES is listed in EQAR, the European Quality Assurance Register for Higher Education.

10. Annual Work Institutions: Research units: Training courses:
universities and schools
3 research organisms
Research units:
600 evaluation committees
Training courses:
700 Bachelor degrees
300 Master degrees
50 doctoral schools

11. The Regional Breakdown of the Evaluation Campaigns
A 5-Year Period

12. Integrated Evaluation
For one institution ( University, Engineer School…)
Evaluation of research units including on site visit
Evaluation reports
Evaluation of institution including on site visit.
Evaluation of training programmes, ( Bachelors, Masters, Doctoral Schools)
Evaluation reports

13. Document Management System
Web based evaluation process (GED)
File and report deposition
Evaluation committee creation
Flags to follow the state and progress of a specific unit evaluation

14. Document Management System (GED)

15. Evaluation of research units

16. Methodological Principles
The criteria defined and procedures implemented by the High Council take into account:
the diversity of the natures and missions of the research units
the diversity of disciplinary fields ( Life Sciences, Social Sciences and Humanities, Physical and Engineering Sciences)
based on peer-assessment (national and international experts)
Evaluation goals:
Oriented towards improvement of unit performance
Combines quantitative data ( scientific production, impact) with qualitative information assessed by experts

17. Evaluation of Research Units: for Who ?
the recipients of the evaluation are:
Research units and researchers
research organisms, Universities, Grandes Ecoles
Students
Local Authorities (regions, etc.)
Ministry for Higher Education and Research and other ministries

18. Evaluation of Research Units
The annual task: evaluation of all State funded research units in specific Regions ( France is divided in 5 evaluation campaigns)
~ 600 research units
> 3000 experts committed every years for the peer review
Covering ALL research bodies supported by public funds, including private universities, architectural and cultural research, and possibly clinical research…

19. Research Unit Evaluation Process
Main steps
The research unit writes a report following a model. This report is deposited on the HCERES document management system (GED).
A HCERES scientific officer is designed to manage the evaluation
An evaluation committee is appointed by HCERES
The committee prepares a draft of the report before the visit
A visit (1, 2 or 3 days ) of the research unit is organised. During the visit, the unit presents its main results and strategy. The committee has meeting with the unit personnel

20. Research Unit Evaluation Process
Main steps (cont’d)
After the visit the committee finalised the report
The report is controlled by HCERES to be sure that it follows HCERES writing rules
The report is transmitted to the unit and its supervising institutions. A summary of the report is published on the HCERES website.

21. The Evaluation Report The report is the main output of the evaluation.
The report should be critical, not descriptive,
The report must contain evaluation details of the research unit at a whole but also for each research team,
The report has to specify strengths, weaknesses and recommendations.

22. Evaluation Report Scientific outputs and quality of research
The evaluation report: the six evaluation items
Scientific outputs and quality of research
Academic reputation and appeal
Interactions with the social, economic and cultural environment
Organization and life of the unit
Involvement in training
Strategy and five-year plan
These six items are used both for the research unit as a whole and for the research groups

23. Evaluation of research units
Synthesis by geographical site:
In addition to the research units evaluation reports, HCERES writes research syntheses :
Goal: To give a schematic view of the particularities of a geographical site based on:
The characterization of working forces in the three domains (Social sciences and Humanities (SH), Life sciences (LS),Physical and Engineering Sciences (PE))
The analysis of evaluation reports in each domain
The synthesis contains two parts
- the workforce
- the synthesis of the evaluation reports
Specific syntheses can be produced by order

24. Breakdown of workforce: Grenoble

25. Breakdown of workforce: Grenoble

26. Open questions
Evaluation is a difficult task. Many questions are still open
How to evaluate interdisciplinary research ?
How to measure the impact of a research ?
How to efficiently use bibliometric data
HCERES working groups and seminars are organized to tackle these questions

27. Conclusion
HCERES, a ‘High Council for Research and Higher Education’ devoted in France to the evaluation of Institutions, research units and training courses
HCERES is an independent authority
HCERES performs an integrated evaluation in each site
HCERES is member of ENQA and EQAR
All the French research units and research groups are evaluated every 5 years