1. P ROCEDURES AND EXPERIENCE IN VERIFICATION OF DATA ON HIGHER EDUCATION INSTITUTIONS IN R OMANIA Paul Serban Agachi Member of the Executive Committee of IREG Observatory University Babes-Bolyai, Cluj-Napoca, Romania 1

2. O UTLINE Why necessary verify HE data? Verification at the level of the university Verification at the level of the system/Ministry Procedures of verification Results 2

3. W HY NECESSARY VERIFY HE DATA ? Since the impact of the rankings on university and national policies is increasing Since the complexity of rankings is increasing Since the aims of using the rankings diversified it is a problem of responsibility of the ranking agencies to give the most reliable rankings 3

4. U-M AP AND U-M ULTIRANK 4 Activity profiles of institutions Multi-dimensional global university ranking Classification U-Map Classification U-Map Multidimenisonal rankings Profile A Profile B... Teaching and learning Research involvement Knowledge exchange Regional engagement Dimen- sion 1 Dimen- sion 2 Dimen- sion 3 Dimen- sion... Multiple excellences International orientation Student profile

5. Selection of criteria and their relative importance Research Education Costs Services Social aspects National context, legislation Financial resources Selection of indicators Validation of data Criticism of existing rankings (I) (Vincke, Dubrovnik, 2009, Bucharest 2012) 5

6. Bibliometrics Quality of data Discrimination between scientific fields Different traditions (journals, books, proceedings, number of authors, duration of research validation) Supremacy of publishing in English Which indicators? (IF, citation index, h index, …) Experts Are there ranking experts? How are they selected? How are the questionnaires structured? How are the answers handled? Criticism of existing rankings (II) (Vincke, Dubrovnik, 2009, Bucharest 2012) 6

7. S UPPOSE 6 UNIVERSITIES C1C2C3C4C5C6 A1,9210,01436,043500,59 B6,006,0754,031500,76 C0,725,01457,050000,85 D1,446,52038,2527501,00 E4,801,81149,548000,87 F3,604,3117534000,73 7

8. R ESULT OF R ANKING C D A F H E B 8

9. C HANGE OF ONE D ATA C1C2C3C4C5C6 A1,9210,01436,043500,59 B8,006,0754,031500,76 C0,725,01457,050000,85 D1,446,52038,2527501,00 E4,801,81149,548000,87 F3,604,3117534000,73 9

10. R ESULT B C D A F E 10

11. TWO CONCLUSIONS Robust algorithms for calculating the positions of the universities in rankings Reliable data similar problems with the allocation of funding for the universities in Romania 11

12. PROPOSAL OF A RANKING AGENCY ROMANIA 12 Steering Committee/Comitet Director WG 1 Methodology WG 2 Elaboration benchmark, data collection and validation WG 3 IT WG 4 Program monitoring Executive director

13. V ERIFICATION AT THE LEVEL OF THE UNIVERSITY Database with the Human Resource of the university (teaching, research, technical, administrative staff) – HR compartment International staff and their status – HR and International offices International students and their status – Registrar and International office National Student Enrollment Registry - Registrar Individual verification of each registration (consistency, nature, values, intervals etc.) Students with the situation incomplete at the end of the academic year Coherence of he output and input data from different consecutive years Unique position in the NSER 13

14. V ERIFICATION AT THE LEVEL OF THE UNIVERSITY IT conditions – IT department Databases with researchers and doctoral students – Research and Doctoral studies compartment Research data- Research compartment Scientometric compartment Internal Database of research activity Citations, IF Comparison with the International Data Bases (Thomson, Scopus) Consistency with the two databases 14

15. V ERIFICATION AT THE LEVEL OF THE UNIVERSITY Database with alumni – Alumni office Databases with alumni at the level of departments/faculty Database with employers at the level of faculties and university Employment situation – in UBB – Center for University Development one of the major problems is that all these data are not correlated and their consistency is not verified by one unique body 15

16. V ERIFICATION AT THE LEVEL OF THE SYSTEM Ministry ( National Student Enrollment Registry ) Individual data for each student Data concerning the university University structure University teaching programs Data concerning schooling capacity figures approved Titles and diplomas 16

17. V ERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS Allocation of funds based on the equivalent student number Allocation of funds based on quality: Teaching staff and its quality Scientific performance: publications, invention patents etc. Extra-budgetary funds and their allocation Quality of social services Internal management of the allocations 17

18. V ERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS Analysis of the students number reported on October 1 of the current academic year relative to the report of the same number on January 1 of the previous academic year Consistency of the statistical data based on the correlation between their value and their significance number of students participants to the pedagogical seminar < number of BA students 18

19. V ERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS Analysis of the students number reported on October 1 of the current academic year relative to the report of the same number on January 1 of the previous academic year Consistency of the statistical data, comparing the enrollment figures on January 1 of the previous academic year with the schooling capacity: variations larger than +/-10%; data referring to the continuing education for teaching staff in the pre-university system – not considered because of the annual fluctuations Variations larger than +/-3%; data referring to period 1 Oct – 1 Jan of the same academic year Identify situations of discordance between situations from Jan 1 of the two consecutive academic years surplus of the declared number of students and schooling capacity 19

20. V ERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS Analysis of the academic staff number reported on October 1 of the current academic year Primary analysis of the consistency Number of academic staff should be < or = to the total number of professors of all grades (full, associate, lecturers etc.) Number of young staff should be < or = to the total number of lecturers, assistants Secondary analysis of the consistency Comparison with the situation of a reference (2008) for the difference larger that +/- 10% Academic tenure staff change; difference in the number of professors of all grades; teaching staff under 35; teaching staff with PhD degree; teaching positions number; vacancies; auxiliary staff change 20

21. V ERIFICATION AT THE LEVEL OF THE SYSTEM CNFIS Analysis of the number of students enrolled and schooling capacity Comparison of the number of students enrolled at October 1 of the academic year and the allocated figure from the Ministry (OMECTS) in the year verified Schooling capacity compared with first year students reported at October 1 of the academic year Comparison between the number of students enrolled in the university, reported in the previous years with the schooling capacity in the same year (e.g. II-nd year students) 21

22. V ERIFICATION AT THE LEVEL OF THE SYSTEM CNCS Analysis of the overall scientific contribution (articles, books, proceedings, patents, prizes, conferences etc.) Analysis and comparison of the data reported by the universities with international databases: ISI Thomson, SCOPUS Result: report for Ministry, ANCS and CNFIS 22

23. V ERIFICATION AT THE LEVEL OF THE SYSTEM ARACIS Analysis of the number of students enrolled and schooling capacity for each program Analysis of the number of academic staff for each program Analysis of the state of the properties of the universities Analysis of the teaching/research space 23

24. CONCLUSION HE data are verified at three levels University Ministry Councils Bad experience with the validation of the data at the last classification New classification/ ranking 24

25. Thank you for your attention 25