Martin Ince Communications Limited The World University Rankings in 2008

Martin Ince Communications Limited The Challenges of University Ranking Presentation by Martin Ince Contributing editor, THE, QS-Apple 2008 Conference Yonsei University, Korea

Martin Ince Communications Limited Times Higher Education Since 1971 Since 1971 Weekly newspaper formerly associated with The Times [of London] Weekly newspaper formerly associated with The Times [of London] Now in magazine format Now in magazine format Group including TES Group including TES Online at since 1994 Online at since 1994www.timeshighereducation.co.uk

Martin Ince Communications Limited Why rank universities? Interest in ranking things and people Interest in ranking things and people - Hospitals - Schools - Local authorities - Rich lists; Britain, world, Asians, footballers - Universities: The Times

Martin Ince Communications Limited Does it matter? Some of these rankings are mainly for entertainment, eg Rich Lists Some of these rankings are mainly for entertainment, eg Rich Lists Others are serious Others are serious - Bad marks for school or hospital - Likewise for a university department - Over 1000 “failing” public bodies in UK – schools, police forces etc

Martin Ince Communications Limited National Rankings The Times The Times - produced by John O’Leary, former editor of THE - Institutions as well as subjects Criteria for subjects include: - Teaching quality - Research quality - Entry standards - Employability

Martin Ince Communications Limited National rankings (2) Criteria for institutions include Criteria for institutions include - Teaching standards - Staff/student ratio - Library spending - Facilities spending - Good degrees - Jobs - Research

Martin Ince Communications Limited The US Comparison US News and World Report “America’s Best Colleges” US News and World Report “America’s Best Colleges” - Mainly about how likely you are to graduate - Also student experience eg class size - However, many other tables eg liberal arts, business, engineering colleges - Likewise McLean’s (Canada) et al

Martin Ince Communications Limited Richness of data Main table in US News has 19 main columns of data Main table in US News has 19 main columns of data Likewise Times Good University Guide Likewise Times Good University Guide How can we do this globally? How can we do this globally? Why would we want to? Why would we want to?

Martin Ince Communications Limited Why world rankings? Long overdue: higher education has always been very international Long overdue: higher education has always been very international Unique position of THE and QS Unique position of THE and QS Universities becoming more global Universities becoming more global Knowledge the real factor in international competitiveness Knowledge the real factor in international competitiveness Increasing desire for comparative information Increasing desire for comparative information

Martin Ince Communications Limited Why world rankings (2)? GATS GATS EU and Bologna EU and Bologna 3 million students outside home country 3 million students outside home country Forecast to be 5 million by 2010 Forecast to be 5 million by 2010 BTA BTA UK as a major source and destination UK as a major source and destination UK as major collaborator UK as major collaborator UK universities opening in China and elsewhere UK universities opening in China and elsewhere

Martin Ince Communications Limited The five flows Goods Goods Services Services Money Money People People Information Information

Martin Ince Communications Limited And Tony says so “In 10 years we will think nothing of students going off to university anywhere in the world” “In 10 years we will think nothing of students going off to university anywhere in the world” Tony Blair at the Labour Party conference, September 2006

Martin Ince Communications Limited World interest Interest from governments – UK Interest from governments – UK Treasury, Lambert EU, Germany EU, Germany Shanghai Jiao Tong Shanghai Jiao Tong HEEACT HEEACT OECD from 2010 OECD from 2010

Martin Ince Communications Limited And of growing importance.. Soft power conveying cultural importance of a nation Soft power conveying cultural importance of a nation Cheaper and more effective than an army Cheaper and more effective than an army Prestigious universities mean international regard Prestigious universities mean international regard Even among non-academics Even among non-academics

Martin Ince Communications Limited Soft power (2) British Council, Goethe Institut, Instituto Cervantes British Council, Goethe Institut, Instituto Cervantes China recognises this: Confucius Institutes, Shanghai rankings China recognises this: Confucius Institutes, Shanghai rankings People like the place where they studied: US, UK, Australia People like the place where they studied: US, UK, Australia They absorb its values They absorb its values And they pay to be there And they pay to be there

Martin Ince Communications Limited How to do it? Audience includes – Audience includes – Internationally mobile students Internationally mobile students Internationally mobile staff Internationally mobile staff Internationally mobile money Internationally mobile money - Focus on: - Teaching - Research - International orientation

Martin Ince Communications Limited What are we measuring? Data on 500 institutions with significant citations Data on 500 institutions with significant citations Have to teach undergraduates Have to teach undergraduates Have to teach in at least two areas Have to teach in at least two areas

Martin Ince Communications Limited Peer review Peer review is the way academic value is measured Peer review is the way academic value is measured We decided to make it the centrepiece of this ranking We decided to make it the centrepiece of this ranking It is the least understood aspect of our work It is the least understood aspect of our work So here is the explanation So here is the explanation

Martin Ince Communications Limited Peer review (2) Total 5,101 over three years Total 5,101 over three years International spread International spread Subject spread Subject spread Active academics Active academics 40 per cent of possible score 40 per cent of possible score

Martin Ince Communications Limited The peers Gathered by QS list building Gathered by QS list building 41 per cent in Europe 41 per cent in Europe 30 per cent the Americas 30 per cent the Americas 29 per cent Asia Pacific 29 per cent Asia Pacific Aggregate no more than three yearsAggregate no more than three years

Martin Ince Communications Limited The question Online survey Online survey The top 30 universities in the topics they know about The top 30 universities in the topics they know about - Arts and humanities - Social sciences - Science - Biomedicine - Technology

Martin Ince Communications Limited Plusses Simple Simple Understandable Understandable Robust because valid sample Robust because valid sample Self-correcting if large enough sample Self-correcting if large enough sample Hard to cheat Hard to cheat

Martin Ince Communications Limited Plusses Faculty-level response can be seen Faculty-level response can be seen Full subject coverage Full subject coverage Especially arts and humanities Especially arts and humanities Also business, education, law Also business, education, law Mitigates citation issue Mitigates citation issue

Martin Ince Communications Limited Minuses Biases include Biases include - Age - Size - Name -Beijing -Loughborough

Martin Ince Communications Limited Employers Another group who know about university quality Another group who know about university quality Innovation in 2005 Innovation in 2005 Mainly private sector Mainly private sector At 10 per cent of total At 10 per cent of total Therefore academics cut from 50 to 40 per cent Therefore academics cut from 50 to 40 per cent Will be improved in future years Will be improved in future years 2007 sample 1, sample 1,471

Martin Ince Communications Limited Employers are: Americas 43 per cent Americas 43 per cent Europeans 32 per cent Europeans 32 per cent Asia Pacific 25 per cent Asia Pacific 25 per cent Again, maximum three years aggregationAgain, maximum three years aggregation

Martin Ince Communications Limited Quantitative measures Aim to measure universities in terms of Aim to measure universities in terms of - Student commitment - Research commitment - International commitment and competitiveness

Martin Ince Communications Limited How to do this Extensive data gathering exercise Extensive data gathering exercise By UK firm QS By UK firm QS Mix of data sources Mix of data sources National National Institutional Institutional Direct contact Direct contact

Martin Ince Communications Limited First quantitative criterion… Staff/student ratio Staff/student ratio Classic measure of commitment to teaching Classic measure of commitment to teaching 20 per cent of final score 20 per cent of final score

Martin Ince Communications Limited How international? Two criteria rated at 5 per cent each Two criteria rated at 5 per cent each Is this somewhere where people want to be? Is this somewhere where people want to be? Staff Staff Students Students - Again raises issues -Visiting scholars? -EU cross-border students? -Doing full courses? -Geography advantage

Martin Ince Communications Limited Citations Classic measure of research quality Classic measure of research quality Use Scopus data Use Scopus data

Martin Ince Communications Limited Citations (2) Citations per staff member to see density of brain power Citations per staff member to see density of brain power Not citations per paper Not citations per paper Avoids medical and US bias Avoids medical and US bias

Martin Ince Communications Limited Citations (3) Well-understood bias Well-understood bias - against non-English publication - against arts and humanities - against national-oriented topics This accounts for the final 20 per cent

Martin Ince Communications Limited Changes in 2007 Scopus: better at Asia Scopus: better at Asia Also more transparent Also more transparent Better at non-English sources Better at non-English sources Deeper Deeper

Martin Ince Communications Limited Changes (2) Z score: Z score: Measure distance from centre Normalise top to 100 Effects outliers such as CalTech, LSE

Martin Ince Communications Limited Changes (3) Harder to cheat Harder to cheat Main effects in South Asia FTEs more rigorously defined These soft changes bigger effect than methodology changes More understanding of odd claims

Martin Ince Communications Limited What did we find? Harvard Harvard The US – 57 in top 200 The US – 57 in top 200 Yale Oxbridge logjam at 2 Yale Oxbridge logjam at 2

Martin Ince Communications Limited But… The top 200 includes universities in 28 states The top 200 includes universities in 28 states US, UK, Australia, Netherlands US, UK, Australia, Netherlands Korea, China, Japan, Taiwan, Singapore, Hong Kong Korea, China, Japan, Taiwan, Singapore, Hong Kong Continental Europe Continental Europe Developing world: Unam, Brazil (2), Cape Town 200= Developing world: Unam, Brazil (2), Cape Town 200=

Martin Ince Communications Limited International commitment US shows up badly US shows up badly HKUST top for international staff, LSE in sixth place HKUST top for international staff, LSE in sixth place London School of Economics top for international students London School of Economics top for international students Yale among few US with international staff Yale among few US with international staff

Martin Ince Communications Limited Peer review Berkeley the winner Berkeley the winner Cambridge, Oxford, Stanford, Harvard popular Cambridge, Oxford, Stanford, Harvard popular Well-liked universities all over the world Well-liked universities all over the world Little evidence of patriotism bias Little evidence of patriotism bias US, UK, Australia, Japan, China, Canada, Singapore dominate the top 20 US, UK, Australia, Japan, China, Canada, Singapore dominate the top 20

Martin Ince Communications Limited Citations Medical faculty is a big plus Medical faculty is a big plus Or major biomedical income Or major biomedical income CalTech the winner by some distance, then Stanford, MIT, ENS CalTech the winner by some distance, then Stanford, MIT, ENS Big country effect is at work here Big country effect is at work here

Martin Ince Communications Limited Staff/student ratio Winner CalTech, US Winner CalTech, US US, French, Swiss, etc institutions all well placed US, French, Swiss, etc institutions all well placed Harvard shows badly here at 15th Harvard shows badly here at 15th Asian and European universities well- placed Asian and European universities well- placed Weak correlation with research – but not zero Weak correlation with research – but not zero

Martin Ince Communications Limited Take home message Small variations don’t matter much Small variations don’t matter much A position can be gained by many combinations of weakness and strength A position can be gained by many combinations of weakness and strength eg, many Asian institutions do well despite scoring low on citations eg, many Asian institutions do well despite scoring low on citations Will improve if more visible publications Will improve if more visible publications

Martin Ince Communications Limited Things that don’t work Library spending Library spending Course cost Course cost Completion Completion Entry standards Entry standards Wealth Wealth Alumni giving Alumni giving

Martin Ince Communications Limited Comparison with Shanghai Jiao Tong Not a newspaper Not a newspaper Nobel + Fields prizes Nobel + Fields prizes Nobel 6/9 in 2007 Fields 4/6 in 21 st century These used twice These used twice Science and Nature Science and Nature Science and Social Science citations Science and Social Science citations 500 rather than our rather than our 200

Martin Ince Communications Limited Comparison with Shanghai Jiao Tong But: But: Ends up looking rather similar near the top Ends up looking rather similar near the top Share 133/200 Share 133/200

Martin Ince Communications Limited And with HEEACT Science focus Science focus Of main policy interest Of main policy interest Avoids Nobel Prize bias Avoids Nobel Prize bias Likewise Science and Nature Likewise Science and Nature Use of ESI means some gaps Use of ESI means some gaps from 540 in THE/QS from 540 in THE/QS

Martin Ince Communications Limited Response More work than writing the thing More work than writing the thing Last year about 30 newspaper articles in Mexico alone Last year about 30 newspaper articles in Mexico alone Interest from media, universities etc across Europe and Asia Interest from media, universities etc across Europe and Asia Less from the US Less from the US Political response – Ireland, Malaysia, Switzerland Political response – Ireland, Malaysia, Switzerland

Martin Ince Communications Limited Job creation More people looking at it than doing it More people looking at it than doing it Leiden, CEPES Leiden, CEPES IREG IREG All highly valuable All highly valuable

Martin Ince Communications Limited Types of response Reject the whole idea Reject the whole idea Complain about their position Complain about their position Think it is about right Think it is about right Wonder how to do better Wonder how to do better Compare Shanghai et al Compare Shanghai et al

Martin Ince Communications Limited Asia Pacific top universities Australia, ANU, 16 Australia, ANU, 16 Japan, Tokyo, 17 Japan, Tokyo, 17 Singapore, NUS, 33 Singapore, NUS, 33 China, Peking, 36 China, Peking, 36 Hong Kong, Chinese University, 38 Hong Kong, Chinese University, 38 New Zealand, Auckland, 50 New Zealand, Auckland, 50 Korea, Seoul National, 51 Korea, Seoul National, 51 Taiwan, Taiwan National, 102 Taiwan, Taiwan National, 102

Martin Ince Communications Limited In the top 200 Australia 12 Australia 12 China 6 China 6 Hong Kong 4 Hong Kong 4 Japan 11 Japan 11 Korea 2 Korea 2 New Zealand 3 New Zealand 3 Singapore 2 Singapore 2 Taiwan 1 Taiwan 1

Martin Ince Communications Limited Future Change.. These are good figures These are good figures Compare France (5), Italy (2), Spain (1) Compare France (5), Italy (2), Spain (1) Japan 11, Germany 12 Japan 11, Germany 12 In the top 500 have more institutions that may emerge In the top 500 have more institutions that may emerge

Martin Ince Communications Limited In top Taiwan Taiwan Thailand Thailand Philippines 2398, 451 Philippines 2398, 451 NZ NZ Malaysia Malaysia

Martin Ince Communications Limited In top Korea Korea Japan Japan Indonesia Indonesia India India China China Australia Australia

Martin Ince Communications Limited Asian improvement Already very international Already very international Well-liked by employers and peers Well-liked by employers and peers Adequate staff numbers Adequate staff numbers Citations: more needed Citations: more needed

Martin Ince Communications Limited How to do better Publish more in the right places Publish more in the right places Be more international Be more international Be better represented academically around the world Be better represented academically around the world Have better employer links Have better employer links Have enough staff to teach your students Have enough staff to teach your students

Martin Ince Communications Limited The future Important for staff, students, parents, management, employers Important for staff, students, parents, management, employers Important for governments – culturally as well as economically Important for governments – culturally as well as economically Important for business Important for business Important globally, eg for the EU Important globally, eg for the EU Management tool: Management tool: - How do we look? - Who to talk to? - What are our ambitions?

Martin Ince Communications Limited Future developments New data New data - Any suggestions? - Subject specific? - Management indicators Refine existing data, eg from employers More global reach, eg Africa New analyses New entrants

Martin Ince Communications Limited The book Published 2006 and again early 2008 Published 2006 and again early 2008 Yours from Yours from institutions including articles on the top 100 and shorter details on the rest 500+ institutions including articles on the top 100 and shorter details on the rest Data in groups after 200 Data in groups after 200

Martin Ince Communications Limited …really the last slide Thanks to Thanks to Organisers of QS-Apple Ann Mroz, editor of THE Ann Mroz, editor of THE Nunzio Quacquarelli, QS Ben Sowter, QS Mandy Mok, QS Asia Mandy Mok, QS Asia Staff of Scopus