1. Dr. Edriss Ali, AGU - Dubai
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Global University Ranking Methodologies and Their Impact on Higher Education Quality
Dr. Edriss Ahmed Ali Associate Professor, College of Engineering & Computing, Al Ghurair University, Dubai, UAE

2. Dr. Edriss Ali, AGU - Dubai
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Outline
Introduction
An Outline of Popular Global Ranking Organizations:
Global Ranking Systems Methodologies
Why Global Rankings do not reflect the actual quality of Institution
Implications of Ranking On Some Universities’ Status
Institutional Quality Indicators
Why Ranking Measures are More Informative than QA Measures:
Proposed Methodology for Ranking
Conclusions & Recommendations

3. Dr. Edriss Ali, AGU - Dubai
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Introduction
A number of famous global ranking systems have recently appeared in the higher education sector, claiming that they compare the education quality of universities and higher education colleges.
This was due to globalization of higher education and the implication they have on so social and economical development.
There are three main levels of Ranking:
National: at country level e. g. US national
Regional: universities existing in a certain region, e. g. Arab World, Asian Region, Latin American, … etc.
Global: at international level e.g. QS, THE, … etc.
Discipline: e. g. Engineering, Business … etc.

4. Evolution of Ranking Systems
National university rankings have been used for decade in the States (1905).
Global rankings first appeared in 2003, when a team of researchers at Shanghai Jiatong University (China) produced the Academic Ranking of World Universities (ARWU) to benchmark Chinese universities with top universities in the World.
In 2004 appeared Times Higher Education – QS World University Rankings (THE-QS).
THE-QS split into Times Higher Education World University Ranking (THE) and QS World University Ranking (QS) in 2010.
In 2009, a multidimensional mapping and ranking project funded by European Union called ‘U-Multirank appeared.
Global rankings then grew over time and coincided with the advance of globalization and importance of higher education for social and economic development and marketization of higher education, and mobility of students and faculty.

5. An Outline of Popular Global Ranking Organizations:
11/22/2018
Dr. Edriss Ali, AGU - Dubai
An Outline of Popular Global Ranking Organizations:
we selected three famous international university rankers; namely:
ARWU: (Academic Ranking of World Universities) by Shanghai Ranking Consultancies,
THE : The Times High Education (World University Ranking),
QS World University Ranking.

6. Data Sources of Global Ranking Systems
Many global rankers, e. g. ARWU, THE , U-Multirank, use ‘Thomson Reuters Web of Science’ and ‘Thomson Reuters Incite’.
Only QS draws its data from SCOPUS, a product of Elsevier, a Dutch-based data broker.

7. Ranking Methodologies
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Ranking Methodologies
Indicator Weights: After selecting indicators, each is given a weight depending (on the opinion of the ranker) on its relative contribution to the HEI quality. Scope: In order to limit the number of HEIs, a criteria is set, such as the size of the institute, number of disciplines it covers, whether under-graduates only or both under and post graduates … etc.

8. Indicators and Weights for ARWU
Criteria
Indicator
Code
Weight
Quality of Education
Alumni of an institution winning Nobel Prizes and Fields Medals
Alumni
10%
Quality of Faculty
Staff of an institution winning Nobel Prizes and Fields Medals
Award
20%
Highly cited researchers in 21 broad subject categories
HiCi
Research Output
Papers published in Nature and Science*
N&S
Papers indexed in Science Citation Index-expanded and Social Science Citation Index
PUB
Per Capita Performance
Per capita academic performance of an institution
PCP
Total
100%

9. Example: QS Ranking System
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Example: QS Ranking System
Methodology of QS World University Rankings[16]
Indicator
Weighting
Elaboration
Academic peer review
40%
Based on an internal global academic survey
Faculty/Student ratio
20%
A measurement of teaching commitment
Citations per faculty
A measurement of research impact
Employer reputation
10%
Based on a survey on graduate employers
International student ratio 5%
A measurement of the diversity of the student community
International staff ratio
A measurement of the diversity of the academic staff

10. Example THE Ranking System
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Example THE Ranking System

11. Comparison of indicators and their weights of three popular Rankers
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Comparison of indicators and their weights of three popular Rankers

12. Dr. Edriss Ali, AGU - Dubai
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Why Global Ranking Systems do not reflect the actual quality of Institution
Indicators used are not inclusive
Indicators used are biased toward a minor set of quality aspects
Relative position in ranking table based on a single average weight
Historical students background not considered
Source of Indictors measurement data
Not all Universities Considered

13. Implications of Ranking On Some Universities’ Status
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Implications of Ranking On Some Universities’ Status
Due to many restrictions imposed by popular ranking systems, many universities in the developing world, especially some Arab region universities, do not find the chance of being considered for ranking, and if considered, are low ranked due to limited resources in fields like research,
Investigation of the list of these universities and their origin shows that most of them are in the relatively rich GCC countries with few exception, where a reasonable amount of fund is allocated to research (King Abdel Aziz Center for Scientific Research in Saudi Arabia).

14. Dr. Edriss Ali, AGU - Dubai
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Arab World Universities in THE Global Raking (The Chronicle of Higher Education) 
Country:
University:
Rank
Saudi Arabia
King Abdulaziz University
King Fahd University of Petroleum and Minerals
King Saud University
UAE
United Arab Emirates University
Lebanon
American University of Beirut
Jordan
University of Jordan
Jordan University of Science and Technology
Qatar
Qatar University
Morocco
University of Marrakech Cadi Ayyad
American University of Sharjah
Egypt
Alexandria University
Cairo University
Suez Canal University
Oman
Sultan Qaboos University

15. Institutional Quality Indicators
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Institutional Quality Indicators
A number of researchers investigated identification a possible set of quality indicators in higher education, among these the work of the Educational Policy Institute of Higher Education Quality Council [10], in which the areas for quality indicators are categorized into five categories as:
Input Indicators (no. students enrolled, no. faculty, FT/PT)
Research Indicators (budget, publications, citations .. Etc.)
Teaching and Learning Indicators (cognitive development, satisfaction surveys both students, employers),
Internal Service Indicators (completion, continuation rates, employment rates … etc.)
Output Indicators (graduate students satisfaction, exit survey, spending on services, libraries, network access … etc.).

16. Why Ranking Measures are More Informative than QA Measures:
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Why Ranking Measures are More Informative than QA Measures:
Parameter
QA Systems
Ranking Systems
Purpose
To guarantee compliance with (minimum) standards
To identify “Excellence” has after commercial purpose
Method of Implementation
Public framework or Independent non-profit organization
Private Companies
Consequences of Outcome
It is mandatory for operation of an Institute or Program
No legal consequences
Type of Input Data
Data have to be collected through teaching and learning process
Use research-related and other criteria for which data are available
Nature of Output Report
Hard to understand by a layman
Results are easily readable and understandable
Relation to quality of Education
Geared towards measuring or indicating education quality
Useful for measuring only certain aspects of quality of education
Cost
Relatively high
Low and sometimes no cost apart from that required for collecting available data

17. Proposed Methodology for Ranking
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Proposed Methodology for Ranking
In order to find appropriate quality indicators that lead to a combined aggregate values for a higher education institutes we first look at main functions of any HEI, which are traditionally grouped into three main areas;
Teaching and Learning,
Research and Other Scholarly activities, and
Community Service (engagement).
All educational and related activities of any HEI can be associated to one of these three main areas.
Hence if quality indicators for each activity related to each of these three areas, an aggregate quality indicator in each can be estimated, and hence an aggregate indicator for the HEI can approximately be estimated.

18. Proposed Methodology for Ranking (The Conceptual Model)
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Proposed Methodology for Ranking (The Conceptual Model)

19. Computation of total grade for each of the three categories (EDU, RES, Com)
𝑬𝑫𝑼» 𝒊=𝟏 𝒏 𝒆 𝒆 𝒊 = 𝑵 𝒆
𝑹𝑬𝑺» 𝒊=𝟏 𝒏 𝒓 𝒓 𝒊 = 𝑵 𝒓
𝑪𝑶𝑴» 𝒊=𝟏 𝒏 𝒄 𝒓 𝒊 = 𝑵 𝒄
𝑵 𝒆 + 𝑵 𝒓 + 𝑵 𝒄 ≤100

20. Conclusions & Recommendations
11/22/2018
Dr. Edriss Ali, AGU - Dubai
Conclusions & Recommendations
The mythologies and type of indicators used by three leading global university ranker have been considered and their shortcomings and the implications they have on educational quality have been identified.
The starting point for ranking should be based on the outcome of Quality Assurance metrics followed by accreditation agencies, with appropriate indicators and their weights being chosen carefully.
A ranking methodology that takes into account all university activities has been proposed.
The need for a regional university ranking system that avoids all flaws of the existing ranking systems, and gives a true measure of HEI quality, based on quality indicators covering all dimensions of higher education quality, has been shown.
This will give a fair judgment of educational quality of institutions, and in-line with AROQA mission statement and goals.