1. INTERNATIONAL TREND ON UNIVERSITY RANKINGS AND ITS IMPACT ON HIGHER EDUCATION Angela Yung Chi Hou, 19 Dec, 2012 Fu Jen Catholic University 1

2. Presentation Outline I.Introduction II.Development of Rankings III.Limitations of Rankings IV.Impact of Rankings V.Outcomes of Ranking VI.Use of Rankings VII.Conclusion 2

3. I. Introduction 1.Three major concerns 2.Meeting challenges to higher education in the 21st century 3.Recipe for higher education 4.World class university 5.Global rankings and world class university 6.Rational of college ranking 3

4. 1.Three Major Concerns for Development of Higher Education in Global Times Accessibility Higher education enrollment rate has been increasing in the past decade Affordability Finance need is not an impediment for eligible students to attend a college Accountability Enormous resources and talent are available in higher education institutions Help students be ready for college, and be equipped to graduate from college 4

5. 2.Challenges to Higher Education in 21st Century Over expansion of higher education 2305 post secondary higher education institutions in China 726 universities and 488 junior colleges in Japan 201 universities in Korea 160 universities in Taiwan 15 in Hong Kong (9/6) 12 in Macau(4/8) Private universities outnumber public universities (except China) Declining birth rate three are below 1.5 Declining financial governmental support 5

6. 3.Recipe for Higher Education - quality assurance and global competitiveness Set up Internal and External Quality Assurance System Establishment of National Accrediting agency Establish Quality Culture on Campus Compulsory audit Enhance international academic competitiveness Launching several Excellence Programs Brain21 (Korea) Center for Excellence COE (Japan) 5 year-50 Billion Program (Taiwan) China 985 project Aiming at establishing world class universities 6

7. 4.What does a world class university look like? In terminology world class universities: top universities striving for “excellence” quality must surpass the expectation of stakeholders Philip Altbach excellence in research, top professors, academic freedom and an atmosphere of intellectual excitement, governance, adequate facilities and funding 7

8. 4.What does a world class university look like? Jamil Salmi (World Bank) based on two rankings (Shanghai and QS) a high concentration of talent (faculty and students) abundant resources to offer a rich learning environment and conduct advanced research favorable governance features that encourage strategic vision, innovation and flexibility, and enable institutions to make decisions and manage resources without being encumbered by bureaucracy 8

9. 5.Relevance between Global Rankings and World Class University characteristics of world class universities are strongly correlated to most indicators used by global rankings nations use global rankings as a basis of building world class universities despite methodological flaws 9

10. 5.Relevance between Global Rankings and World Class University top administrators at leading universities use global rankings to achieve the short term and long term strategic plans, not just to boycott them Minnesota’s initiative  become one of the top three research institutions in the world Taiwan National University  “Moving into the top 100” at its 80th anniversary Baylor University  one of the U.S. News Top 50 by 2012 10

11. 6. Rational of College Ranking Higher education expansion Resources allocation Accountability Benchmarking Marketization in higher education 11

12. II. Development of Rankings 1.National College Ranking 2.Global Ranking 3.Types of Rankings 4.Scoring Methods 5.Five Global Rankings 6.Criteria & Indicators of Rankings 12

13. 1.National College Ranking U.S. News & World Report The most influential college ranking – “American Best Colleges” published by U.S. News & world Report in 1983 Maclean’s, The Times, CHE, etc. 13

14. 2.Global Ranking intense international competitions global college rankings have drawn international attention 14

15. 3.Types of Ranking By region by country/continent/worldwide By field / program such as Engineering/social science/humanities By subject i.e., Biology/ Math College guide Princeton review 15

16. 4. Scoring Methods The indicators were weighted at a certain ratio and the scores were aggregated to rank each college. The top one university received highest points while the scores for the remaining schools descended accordingly. 16

17. 5. Five Global Rankings Academic Ranking of World Universities Shanghai Jiao Tong University, 2003 World University Ranking QS, 2004 Webometrics Rankings of World Universities Spanish National Research Council, 2004 Performance ranking of scientific papers of world class universities HEEACT, 2007 World University Ranking The Times Higher Education, 2010 17

18. 6.Criteria & Indicators of Ranking 2011 ARWU Ranking 2011 QS Ranking 2011 Webometrics Ranking 2011 THE Ranking 2011 HEEACT Ranking 18

19. 6. 2011 ARWU Ranking 分類指標分項指標權重操作型定義 分類指標分項指標權重操作型定義 教育品質 畢業校友 (Alumni) 10% 獲得諾貝爾及菲爾茲獎校友人數。得分高低依畢業 年度，每往前推十年，降低 10% 權重。如 2001 年以 後每人以 1 人計算； 1991-2000 年每人以 0.9 人計算 ， 依次遞減。 教師品質 獲得重要獎項人數 (Award) 20% 獲得諾貝爾及菲爾茲獎教師人數。得分高低如上。 如 2001 以後獲獎者每人以 1 人計算； 1991-2000 年每 人以 0.9 人計算，依次遞減。獲獎人同時署名兩單位， 各計.05 人。諾貝爾共同得獎者，以獎金比例分配權 重。 論文高度被引用研究人數 (HiCi) 20% 在 21 個科學領域中，論文名列 ISIHighlyCited.com 高 度引用之教師人數。 研究成果 《自然》與《科學》兩期刊論 文發表篇數 (N&S) 20% 2006 至 2010 年間，一所大學在《自然》與《科學》 兩期刊論文發表篇數，不包含評論與快訊。 SCI 與 SSCI 論文數 (PUB) 20% 2010 年，一所大學發表論文被 SCI 及 SSCI 收錄的數量。 機構規模機構規模（專任教師數） (PCP) 10% 以上五項總分 / 專任教師數

20. 6. 2011 QS Ranking 評比內容權重分項指標 研究 (Research) 60% 學術聲譽 (Academic Reputation) 教師平均論文被引用數 (Citations per Faculty) 就業力 (Employability) 10% 就業聲譽 (Employer Reputation) 教學 (Teaching) 20% 師生比 (Faculty/Student Ratio) 國際化 (Internationalization) 10% 國際教師比例 (International Faculty Ratio) 國際學生比例 (International Student Ratio)

21. 6.2011 Webometrics Ranking 21 IndicatorsDefinitionCovergaeSourcesWeight IMPACTNumber of backlines Number of back domains CurrentMajestic SEOMajestic SEO and ahrefs ahrefs 50% PresenceNumber of web p ages CurrentGoogle20% opennessNumber of papers (pdf, doc, docx, ppt) 2007-2010Google scholar 15% Excellence10% most cited papers in their respective scientific fields 2003-2010Scimago15%

22. 6. 2011 THE Ranking 評比內容權重分項指標 分項指 標權重 教學－ 學習環境 (Teaching -the learning environment)the learning environment 30% 教學聲譽 (Reputational survey-Teaching) 15% 教師平均指導博士畢業生數 (PhD awards per academic) 6% 教師平均教授大學生數 (Undergraduates admitted per academic) 4.5% 授予博、學士學位比 (PhD-to-bachelor’s ratio) 2.25% 教師平均分配院校收入 (Income per academic) 2.25% 研究－ 數量、收入與聲譽 30% 研究聲譽 (Reputational survey-Research) 18% 研究收入 (Research Income) 6% 教師與研究員平均論文篇數 (Papers per academic and research staff) 6% 論文引用－研究影響 30% 平均論文引用數 (Average citations per paper) 30% 國際化－職員與學生 (International outlook -staff and students) 7.5% 國際對本地生比 (Ratio of international to domestic students) 2.5% 國際對本地教職員比 (Ratio of international to domestic staff) 2.5% 國際合作論文比例 (Proportion of internationally coauthored research papers) 2.5% 業界收入－創新 (Industry income - innovation) innovation 2.5% 研究人員自業界之平均研究收入 (Research income from industry per academic staff) 2.5%

23. CriteriaIndicatorWeight Research productivity Number of articles in the last 11 years (2000- 2010) 10% 20% Number of articles in the current year (2010) 10% Research impact Number of citations in the last 11 years (2000- 2010) 10% 30% Number of citations in the last 2 years (2009- 2010) 10% Average number of citations in the last 11 years (2000-2010) 10% Research excellence h-index of the last 2 years (2009-2010) 20% 50% Number of Highly Cited Papers (2000-2010) 15% Number of articles in high-impact journals in the current year (2010) 15% 6. 2011 HEEACT Ranking 23

24. III. Limitations of Rankings 1.Characteristics of 5 major Global Rankings 2.Methodological Limitations 24

25. 1.Characteristics of 5 Major Global Rankings ARWUQSWebometricsHEEACTTHE Established year 20032004 20072009 InstitutionAcademic institution Mass media/Private Education consulting firm Governmental research unit QA AgencyMass media GoalAcademic competition Profit makingAcademic sharing BenchmarkingAcademic Competition Number of indicators 664813 Indicator category Research output/ learning input Research output / Reputation survey / learning input Web size/ research output/ reputation Research output Teaching/ Research Output /survey Data sourcesDatabaseSurvey/ database/ institution databaseDatabaseSurvey/ database/ institution Outcomes Presentation Only Top 100 of 500 institutions are shown in numerical orders Top 400 are shown in numerical orders Top 1000 in numerical order Top 500 in numerical order Top 200 are Shown in numerical orders TransparencyHighly mediumMediumHighly medium Medium 25

26. 2.Methodological Limitations of Global Rankings Reductionism / Simplicity Research focus Unfair for humanities, arts and social science fields English domination Arbitrary selection of indicators and weightings Data quality 26

27. IV. Impact of Rankings 1.Outcomes of Rankings 2.Popular Use by Stakeholders 3.Two Major Reports 4.Performance in Asia 27

28. 1.Outcomes of Rankings US and UK institutions are on the top Asia is on rise, particularly those with Excellence policy 28

29. 2.Popular Use of Global Rankings by Stakeholders Students to decide where to study Governments to know where to invest Scientists to know where to work Institutions to know where they stand and whom they can partner with 29

30. 3.Two Major Reports OECD survey in 2007 showed: over 50 % of respondents: rankings has a positive impact on the institution’s reputation (student recruitment, academic partnerships and collaborations and staff morale) majority of the institutions incorporated the outcomes of rankings into strategic planning processes at all levels 70 % wanted to be in the top 25 internationally 30

31. 4.Two Major Reports OECD survey in 2007 showed: an on–line UK study focused on English Universities ‘ attitudes toward rankings rankings reflect the views of what properties a good university should develop that influenced the institutional and governmental polices a high level of agreement that the reputation of an institution might be affected by rankings many institutions further down in the rankings do not care too much about global rankings 31

32. 5. Average Number of top 500 Universities of Japan, China, South Korea and Taiwan by three rankings Country ARWU (2004~2012) HEEACT (2007~2011) QS (2007~2012) Ave No Japan29.5629.6025.6728.27 China16.4414.2012.3314.33 South Korea9.009.2011.509.90 Taiwan6.225.408.506.71 Hong Kong5.00 5.675.22 India2.00 8.004.00 32

33. V. Ranking Outcomes 1.ARWU / QS/ HEEACT/ THE Ranking 2.Number of papers and internationalization in China, Korea, Taiwan and Japan 33

34. Top 20 Universities in the Four Rankings (2/2) University Name HEEACT (2011) ARWU (2012) THE (2012) QS (2012) University of California - San Diego15 --- The University of Tokyo1620--- University of London - University College London 17---174 Duke University18--- 20 Yale University1911 7 Imperial College London20---86 California Institute of Technology---6110 Princeton University---759 University of Chicago---998 Cornell University---132014 University of Wisconsin - Madison---19--- ETH Zürich - Swiss Federal Institute of Technology Zürich --- 1513 McGill University--- 18 34

35. Top 20 Universities in the Four Rankings (2/2) University Name HEEACT (2011) ARWU (2012) THE (2012) QS (2012) University of California - San Diego15 --- The University of Tokyo1620--- University of London - University College London 17---174 Duke University18--- 20 Yale University1911 7 Imperial College London20---86 California Institute of Technology---6110 Princeton University---759 University of Chicago---998 Cornell University---132014 University of Wisconsin - Madison---19--- ETH Zürich - Swiss Federal Institute of Technology Zürich --- 1513 McGill University--- 18 35

36. Figure 1: The number of the paper published on SCI and SSCI journals in China, Taiwan, Japan, South Korea 36

37. Figure 2: Number of papers in Nature & Science in seven years in China, Taiwan, Japan, South Korea 37

38. Figure 3: Number of international students in China, Taiwan, Japan, South Korea 38

39. VI. Use of Rankings 1.Findings in rank Mobility 2.Rank Differences and moving UP in 4 Global Rankings 3.Berlin Principles and Ranking Audit 4.Future Development 5.CHE Excellence Ranking and Research Ranking 6.College Navigator in Taiwan 39

40. 1.Some Findings in Rank Mobility of Global Rankings Hou, Yung-chi, & M. Robert (2011). An Analysis of Positions Mobility in Global Rankings: Making Institutional Strategic Plans and Positioning for Building World Class Universities. Higher Education Research & Development (SSCI). To explore the major factors of rank mobility in 4 major rankings. 40

41. 2.Rank Differences and moving Up in 4 Global Rankings Comparison among 4 Global Rankings by positions rising Implication of 4 Global Ranking on making institutional strategic plans 41

42. 2.Comparison among 4 Global Rankings by Positions Rising ARWU QSWebmetricsHEEACT Cluster one1-17 1-30 1-391-19 Cluster two20-45 Over 30 40-9920-45 Cluster threeOver 46 X Over 100Over 46 total number of positions moving ups 218(500) 170 (400) 242 (500)231(500) Highest ranks moving up94 12521282 42

43. 2.Flow Chart of Implication of 4 Global Ranking on Making Institutional Strategic Plans Webometrics RankingQS RankingsARWU/Shanghai Ranking HEEACT Ranking: Used to inspect the quality and quantity of FACUTLY publications annually Short term(3-5 years) Mid-term 5-15 years Long-term(15~30years) Technology/Internet International Reputation Academic Excellence 43

44. 3.Berlin Principles and Ranking Audit International Ranking Expert Group (IREG) founded in 2004 by the UNESCO European Centre for Higher Education (UNESCO-CEPES) in Bucharest and the Institute for Higher Education Policy in Washington, DC. 44

45. 3.Berlin Principles and Ranking Audit It is upon this initiative that IREG’s second meeting (Berlin, 18 to 20 May, 2006) has been convened to consider a set of principles of quality and good practice in HEI rankings—the Berlin Principles on Ranking of Higher Education Institutions. IREG-5 in Oct, 5-7, 2010 proposed “Ranking Audit” 45

46. 3.Berlin Principles: What Rankings and League Tables should Consider To ensure the quality of rankings: research methods, indicators, data quality, transparency, varying user’s interests, etc. 4 major principles Purposes and Goals of Rankings Design and Weighting of Indicators Collection and Processing of Data Presentation of Ranking Results 19 criteria of audit published by IREG 46

47. 4.Future Development for Rankings Field/subject based ranking Varying ranking providers More interactive, multi-dimensional, personalized CHE European Excellence Ranking College Navigator in Taiwan Web-based Benefit student mobility Student survey 47

48. 5.CHE Excellence Ranking and Research Ranking Multi-dimensional global ranking that will be based on the CHE ranking approach. Results of the feasibility study will be available in mid-2011. One aspect will be the development of a concept to introduce a web-based tool for personalized rankings for particular target groups on a global scale. 48

49. 49

50. 50 College Navigator in Taiwan Published in 2009 Goal lead to a match between the student and the institution or the program that they’re most interested Selection of Institutions 69 4-year colleges and universities evaluated by HEEACT from 2006 to 2010. 77 University of Technology and Technical College 50

51. Research Team 51  3 professors/ one international consultant / one full time staff/ one IT designer / 7 graduate students

52. 6.College Navigator in Taiwan-Home Click here to start 52

53. College Navigator in Taiwan- Step 1: Indicators 53 Step 1: Choose the indicators

54. College Navigator in Taiwan- Step 1: Indicators 54 Step 1: Choose the indicators

55. College Navigator in Taiwan- Step 2: Weighting 55 Step 2: Give each indicator a weight

56. College Navigator in Taiwan- Step 3: Preference 56 Step 3: Decide the Universities you want to compare Way 1: Narrow down your choices Way 2:Choose Universities

57. College Navigator in Taiwan- Step 4: Result 57 The indicators you choose Our system will analyze the rank of the universities according to the indicators and weights you decide The performance of each university

58. College Navigator in Taiwan- Step 5: General l information 58

59. 59

60. Hou, Angela Yung-chi, Morse, R., & Shao, Y. J. E. （ 2012 ）. Is There a Gap between Students’ Preference and University Presidents’ Concern over College Ranking Indicators? : A Case Study of “College Navigator in Taiwan”, Higher Education ( in press) (SSCI) (2010 Impact Factor 0.823). ISSN: 0018-1560 60

61. Table 6 Mean and STD by items Figure 2: Scatter Plot of level of Importance over Criteria and Indicators 61 Categories Mea n ST D 95% Confidence Interval UpperLower Institutional policy making3.95 0.6 1 4.143.76 Staff and faculty recruitment3.93 0.6 7 4.143.73 Research output4.06 0.7 6 4.303.83 Resources allocation3.82 0.6 9 4.043.61 Student services and learning environment 3.86 0.7 2 4.093.62 System operation3.93 0.7 5 4.163.69

62. 62 Focus group’ opinions Most popular indictors Institutions’ attitude Graduate rate Academic survey * (higher) Institutional policy making ( rank 2) Total amount of equipment per student Expenditure per student Resources allocation (Rank 6) Expenditure per student Enrollment rate Student services and learning environment (Rank 5) Number of international academic awards earned by students within last 3 years students within last 3 years Faculty-student ratio* (higher) Student services and learning environment(Rank 5) Total library holdings per student/ Total grant from collaborations between of University and industry Graduation rate* (lower) Student services and learning environment (Rank 5) Number of national academic awards by students (higher) Student services and learning environment (Rank 5) Total NSC Research grants Total library holdings per student** (lower) Resources allocation (Rank 6) Proportion of faculty members with Ph.D. Total NSC grants per faculty** (lower) Research output (Rank 1) Number of national academic awards earned by students within last 3 years Proportion of full-time faculty Faculty Resource (Rank 3) Average piece of collaborations between of University and industry per full-time faculty Proportion of professors with Ph.D. Table 8 Comparison of rank of importance of the indicators among focus students, general users and institutions

63. VII. Conclusion 1.Conclusion 2.Final Questions 3.Yes and No Answers 4.How Ecosystem Influences Top Research Universities 63

64. 1.Conclusion To achieve a good rankings is becoming more and more important. Global rankings are increasingly being used as a tool for building world class universities and pursuing academic excellence. 64

65. 2.Final question raised by the presentation To what extent can a world class university be replicated by using the factors highlighted in a ranking model? How can it be done? The answer is both “yes,” it can be replicated, and “no,” it can’t be. 65

66. 3.Yes and No Answers Rankings can only provide very rough guidance and clues to institutions on which road to take to achieve academic excellence. Not taken into account, but very crucial: a clear vision institutional features favourable governance sufficient resources 66

67. 4.Understanding How the Ecosystem Influences the Performance of Top Research Universities by Jamil Salmi 67

68. “THERE IS NO SINGLE ROAD TO EXCELLENCE” by Jamil Salmi (2010) Coordinator, Tertiary Higher Education, World Bank 68

69. Thank you for your attention! Questions and Comments 69

70. References *Hou, Angela Yung-chi (2012). Mutual Recognition of Quality Assurance Decisions on Higher Education Institutions in Three Regions-A Lesson for Asia. Higher Education, 64:911-926. (SSCI) (2011 Impact Factor 1.016). ISSN: 0018-1560 *Hou, Yung-chi, Morse, R., & Shao, Y. J. E. （ 2012 ）. Is There a Gap between Students’ Preference and University Presidents’ Concern over College Ranking Indicators? : A Case Study of “College Navigator in Taiwan”, Higher Education, 64:767–787. (2011 Impact Factor 1.016) Hou, Angela Yung-chi (2012 Quality in Cross-Border Higher Education and Challenges for the Internationalization of National Quality Assurance Agencies in the Asia-Pacific Region –Taiwan Experience. Studies in Higher Education. (online) (SSCI) *Hou, Yung-chi., Ince, M., & Chiang, C.L. (2012). A Reassessment of Asian Excellence Programs in Higher Education – the Taiwan Experience. Scientometrics. 92, 23-42 (SSCI). Print ISSN: 0138-9130. (2011 Impact Factor 1.966) *Hou, Yung-chi. （ 2011 ）. An Analysis of Positions Mobility in Global Rankings: Making Institutional Strategic Plans and Positioning for Building World Class Universities. Higher Education Research & Development (SSCI ） (online) ISSN: 0729-4360(2011 Impact Factor 0.901) *Hou, Yung-chi. （ 2011 ）. Impact of excellence programs on Taiwan higher education in terms of quality assurance and academic excellence, examining the conflicting role of Taiwan’s accrediting agencies. Asian Pacific Educational Review, 13, 77-88 (SSCI). ISSN: 1598-1037 *Hou, Yung-chi. （ 2011 ）. Quality Assurance at a Distance : International Accreditation in Taiwan Higher Education, Higher Education, 61(2), 179–191 (SSCI)