Ben Sowter Head of Research QS

Ben Sowter Head of Research QS
Times Higher - QS World University Rankings Methodology & Europe’s Performance Ben Sowter Head of Research QS Main Title Slide There are two faint buttons at the bottom left of the screen throughout the presentation. These link to the “Extras Menu” which consists of some added detail on a number of matters outside the main flow of the presentation. These slides may be useful in answering some frequently asked questions but the presenter should familiarise themselves both with the content and functionality of the buttons first: the top one leads to the extras menu and the bottom one to the “Last Slide Viewed” University of Edinburgh 10 March 2008

Speaker Introduction Graduated in Computer Science from University of Nottingham in 2000 Union Prize Chair, Nottingham University Debating Society Spent 3 months on work placement in northern India in 1999 Worked for international student charity – AIESEC – for two years after graduation 2nd Year as UK National President Operated on international strategic development team Joined QS in 2002

Objectives Establish QS background & credentials
Explain research methodology New developments Understand limitations of rankings Provide insight into performance enhancement specific to improved ranking Collect feedback and input Objectives A serious objective of these presentations has been to establish the credibility of QS as a qualified operator of this kind of product that is the reason for quite a number of slides in the presentation being focused on seemilngly unrelated aspects of the QS business.

Introduction QS Why World University Rankings? THE – QS Collaboration
Our Approach Results UK & Ireland in the Rankings Progress in the Rankings This is a basic session flow – tell them what you’re going to tell them

QS A brief introduction
This section of the presentation is designed to provide some background and explanation to the nature and work of QS

Founded 1989/1990 Nunzio Quacquarelli
MBA Wharton The MBA Career Guide (now QS Top MBA Career Guide) MBA Events Grad School/Postgraduate Events Careers Software Scholarships Nunzio was the founder of QS whilst studying for his MBA at Wharton The MBA Career Guide was the first product Expanding into MBA events (QS World MBA Tour) From there to Masters & PhD fairs in the form of the QS World Grad School Tour Careers services (QS Global-Workplace – job site for MBA grads, QS Leadership Career Forums – careers events focused on diversity) Software (QS TopMBA Search & Scorecard – personalised ranking system for MBA programmes, QS Top Apply – fully featured, customisable application management system) Scholarships – since 2004 (foundation of the QS Education Trust – see later slide)

QS Mission… Our Channels
…to enable motivated people around the world to fulfill their potential, by fostering international mobility, educational achievement and career development Our Channels primary research leading-edge editorial & publications developmental events web solutions Speaks for itself – may wish to emphasise that whilst QS is a commercial operation at our centre we do have a mandate for helping people

www.topuniversities.com The website...
This slide contains a number of screenshots. It is recommended that the presenter previews how the slide runs beforehand. Homepage Rankings Results Example (Advanced) School Profile Grad School Section

www.topuniversities.com Over 1,180,000 visits in 2007
Over 6,000 visits per day in 2008 Increasingly strong prominence in Google & Yahoo searches Ranked 53,626 in Alexa Home of the THE – QS World University Rankings Institution Profiles Detailed study abroad information Numbers for this slide are changing all the time but conservative projections suggest over 1 million visits in 2007 As can be seen on the previous slide there are a lot of advertising opportunities on the site.

Top Universities Guide
New revised and updated edition available now Contains much information not available on web Book is 512 pages, distributed in major bookshops including Amazon

Other QS Initiatives QS World Grad School Tour
QS Top Grad School Guide QS World MBA Tour QS Top MBA Career Guide QS APPLE – Korea July 2008 QS Top Apply QS Education Trust QS Intelligence Unit

QS Offices & Events

Why rank universities? Interest in ranking things and people Hospitals
Schools Local authorities Rich lists; Britain, world, Asian British, footballers Universities: The Times / US News etc… There is a global public fixation with rankings of things – other examples include the top 10 most downloaded celebrities for example The US News & World Reports Ranking has been in existence since 1983, The Times Ranking since the early 90s.

Why World Rankings? Higher education becoming more global
Knowledge the key driver of international competitiveness Increasing desire for comparative information Unique positions of THE and QS as international and independent experts in higher education Raises awareness of all 500+ universities involved in the project

International Study Trends
Worldwide, the demand for international education is forecast to increase rapidly In 2003, an estimated 3.1 million students studied internationally There are likely to be in excess of 5.8 million international students in 2010 Two-thirds of this global demand will be Asians US domination of international student recruitment has begun to falter – tight Visas, increased competition (IIE, NAFSA, CCE) Bologna Accord will create an ‘explosion in English- language based postgraduate courses in Europe’ It is no coincidence that two independently conceived and developed rankings of world universities appeared within 12 months of each other… the Webometrics online ranking actually makes 3. A global environment of readiness appears to emerge with any university that hopes to be successful looking to globalise. These few slides are designed to contextualise the emergence of rankings. The presenter may have their own statistics, anecdotes or examples that reinforce this aspect of the presentation.

Student Mobility on the Rise

International Students Abroad
This chart should animate automatically. If it doesn’t you may either have a fixed chart or an empty chart and need to click for each bar on the chart. It would be a good idea to test how this slide works before the presentation. Source: IIE – Institute of International Education

Government funding of tertiary education being cut (per capita) in most countries around the world (IFC) Many governments targeting international students as a source of tertiary sector funding Australia has estimated that international students are more important to their economy than manufacturing & mining The UK Government estimates that 270,000 students contribute $3 billion in fees & a further $3 billion in other spending IFC = International Finance Corporation – Part of the World Bank Group

By 2010 The European Commissions Bologna Accord will make room for over 500,000 first degree graduates to study in other EU nations for a Masters degree By 2016 In the USA, recent Abraham Lincoln Study Abroad legislation aims to encourage 1 million students to study overseas By 2020 Over 3 million Asians are expected to study outside their home country (Source: IDP – British Council)

Professor Shih, President National University of Singapore
World University Rankings can help differentiate the active, from the dormant…volcanoes! Professor Shih, President National University of Singapore

Academic Ranking of World Universities
Operated by Shanghai Jiao Tong University in China Focuses principally on research output Nobel Prizes & Fields Medals Citations HiCis Strongly correlated with educational reputation Historical dependency Available on

Shanghai Jiao Tong Criteria
Indicator Weight Quality of Education Alumni of an institution winning Nobel Prizes and Fields Medals 10% Quality of Faculty Staff of an institution winning Nobel Prizes and Fields Medals 20% Highly cited researchers in 21 broad subject categories Research Output Articles published in Nature and Science* Articles in Science Citation Index-expanded, Social Science Citation Index Size of Institution Academic performance with respect to the size of an institution

Shanghai Jiao Tong – Issues?
Strengths? Weaknesses? All data is easily verifiable First global ranking established Very mechanical methodology Nobel prizes look backwards Citation data is historic Citation measures bias towards sciences and bio-sciences Overall bias towards US universities Just because we do it differently, does not mean that the SJTU ARWU holds no value… in fact it is titled the Academic Ranking of World Universities so it is perhaps appropriate that they take a more directly academic approach to their work. We attempt to take a more holistic approach… both approaches have value, they are just measuring different things.s

Webometrics Ranking of World Universities
Focuses on web profile Research productivity using Google Scholar Search Engine performance Site popularity Good practices in web management Fully automated Tracks in the realm of 14,000 institutions Prone to anomaly A university domain change can be disastrous Available on

Webometrics Criteria 20% 15% 50%
Indicator Weight Size Number of pages recovered from four engines: Google, Yahoo, Live Search and Exalead. 20% Rich Files After evaluation of relevance to academic and publication activities and considering the volume of the different file formats, the following were selected: Adobe Acrobat (.pdf), Adobe PostScript (.ps), Microsoft Word (.doc) and Microsoft Powerpoint (.ppt). These data were extracted using Google, Yahoo Search, Live Search and Exalead. 15% Scholar Google Scholar provides the number of papers and citations for each academic domain. These results from the Scholar database represent papers, reports and other academic items. Visibility The total number of unique external links received (inlinks) by a site can be only confidently obtained from Yahoo Search, Live Search and Exalead. 50%

World University Rankings

THE - QS World University Rankings
Began in 2004 Collaboration between Times Higher Education (THE) & QS THE running since 1971 THE is a weekly newspaper distributed to academics in the UK and internationally Formerly associated with The Times Published annually in the fall

THE – QS World University Rankings
THE METHODOLOGY

Our Approach Four key themes, as basis for comparison:
Research Quality Graduate Employability International Outlook Teaching Quality

Ranking Criteria & Weights
*All decisions regarding the allocation of weightings are the responsibility of the Times Higher Education Supplement Criteria Indicator Brief Description Weight* Research Quality Peer Review Composite score drawn from peer review (which is divided into five subject areas). 3,703 responses. 40% Citations per Faculty Score based on research performance factored against the size of the research body 20% Graduate Employability Recruiter Review Score based on responses to recruiter survey. 738 responses 10% International Outlook International Faculty Score based on proportion of international faculty 5% International Students Score based on proportion of international students Teaching Quality Student Faculty Score based on student/faculty ratio

Selection of Initial List
Began in 2004 with list of the top 500 world institutions by research impact Excludes single-faculty institutions Excludes postgraduate-only institutions Added on recommendation from certain countries where English publication culture is not strong (e.g. Germany) Each year list is re-evaluated Omitted institutions are welcome to make a case for inclusion with supporting evidence for their inclusion relative to an included institution 566 universities considered in 2007

Sir Richard Sykes Rector, Imperial College
Peer review is an effective way to evaluate universities. It takes smart people to recognise smart people Sir Richard Sykes Rector, Imperial College

Peer Review of Research Output
Peer review is the centrepiece of this ranking 5,101 respondents International spread Weighted Results Subject spread Arts & Humanities Engineering & IT Life Sciences & Biomedicine Natural Sciences Social Sciences Overall score built up from each faculty area Active academics 3 year “latest response”

Peer Review – Selection of Peers
3 year latest response Invitation to previous respondents World Scientific Database (180,000) Singapore Headquarters Mardev Database (12,000+) Part of Reed Elsevier UK Headquarters Academic Sign-up facility

Observations: Peer Review
Harvard second to Berkeley Oxford and Cambridge Well-liked universities all over the world Little evidence of patriotism bias UK and North America dominate the top 20 Indonesia cf Australia

The Recruiter Review

Recruiter Review Another group who have knowledge of university quality Key relevance to “graduate employability” criteria Introduced for 2005 rankings Sourced from QS database and media and university referrals (over 120 lists received for 2007) Improved response for 2006 & 2007 Continuing development focus for 2008

Observations: Recruiter Review
UK Strong but less so than in previous years Some institutions with particular strengths do well Bocconi, HEC Indonesia cf Australia

Quantitative Measures

Quantitative Measures
Research Quality citations/faculty Teaching Quality faculty student ratio International Outlook % international faculty and students Graduate Employability ??

Observations: Staff/Student
Winner – Caltech US, French, Swiss, Netherlands etc institutions all well placed Both Asian and European amongst the well- placed Weak correlation (0.21) with research – but not zero Teaching – research link

Observations: International
US shows up badly HKUST top in staff London School of Economics top in students Caltech among few US with international staff Staff Students Europe and Australia

Citations per Faculty Score
Classic measure of research quality Citations per staff member (not per paper) 5 year score (changed from 10 in 2006 to increase “topicality”) Source: Scopus

Observation: Citations
Medical faculty is a big plus Or major biomedical income CalTech the winner, then Stanford & MIT Big country effect For reasons mentioned above Explain big country effect

Z-Score Aggregation Method used in various domestic rankings such as The Times (UK) Stabilise data year on year More accurate application of weightings throughout the sample Smoothes any anomalies that remain Essentially smoothes the curve for each individual indicator

The Effect of Z-Score Application on Rankings Indicators

2007 Indicator Correlations
Peer Review 0.59 Recruiter Review Strong 0.28 International Faculty Weak 0.35 International Students Moderate 0.32 Student Faculty 0.45 Faculty Citations 0.33 0.37 0.27 0.12 0.66 0.15 0.23 0.19 0.21

Effects of 2007 developments…
Online System More thorough and complete data all the way down the list Less need for averages Peer Review Countries with disproportionately high response but a small number of institutions lose unnatural advantage FTE Data Universities with particularly high or low numbers for part-time will be influenced Scopus Less pronounced bias towards US and English speaking world Z-Score aggregation Less advantage for the best institutions in lower weighted indicators Increased stability year on year

NOTE In understanding and using any ranking, it essential to develop a full understanding of what they are measuring THE & QS are committed to: Being open with our methods and data wherever possible Engaging with universities and other stakeholders to help them understand the rankings Providing as much qualitative information in support of the rankings as possible

World University Rankings - Issues?
The ranking has limitations. Lack of data on: Teaching quality Student satisfaction Investment in infrastructure The ranking relies on the best comparable data available

THE RESULTS

Comparing Results THE – QS SHANGHAI WEBOMETRICS 1 Harvard 2= Cambridge
2= Yale 2= Oxford 5 Imperial 6 Princeton 7= Caltech 7= Chicago 8 UCL 10 MIT SHANGHAI Harvard Stanford Berkeley Cambridge MIT Caltech Columbia Princeton Chicago Oxford WEBOMETRICS MIT Stanford Harvard Penn State Berkeley Michigan Wisconsin Minnesota Illinois Cornell

Key Findings Harvard The US – 56 in top 200
Harvard’s lead the smallest yet Oxford and Cambridge also well placed But that’s only part of the story

Key Findings The top 200 includes universities in 28 states
US, UK, Canada, Australia, Netherlands Korea, China, Japan, Singapore Thailand, Malaysia Continental Europe Developing world small but improving (1 in 2004, 2 in 2005, only UNAM in 2006, 3 in 2007) Cf cars or computers This shows us that universities really are an international industry in which quality can be measured by simple and robust means Large english speaking but also… Sao Paulo, Unam

Where are the top universities?
Region/Country 2007 2006 2005 2004 N. America 14 (11) (12) 23 (25) (23) 43 (36) (35) (38) Europe 4 (5) 12 (13) 35 (41) (37) Asia/Australia 3 (4) (3) 15 (14) 22 (28) USA 13 20 (22) (20) 37 (33) (31) UK 8 (8) 19 (15) Australia 1 (1) 6 (6) (7)

Consistency of Results
2007 2006 2005 2004 1 Harvard 2 Cambridge MIT Berkeley 3 Yale Oxford 4 Caltech 5 Imperial Stanford 6 Princeton 7 Chicago 8 9 UCL 10 Ecole Polytechnique ETH Zurich

The Results 2007 2006 NAME COUNTRY SCORE 1 HARVARD University
United States 100 2= 2 University of CAMBRIDGE United Kingdom 3 University of OXFORD 4= YALE University 5 9 Imperial College LONDON 99 6 10 PRINCETON University 7= 7 CALIFORNIA Institute of Technology (Caltech) 11 University of CHICAGO 25 UCL (University College LONDON) 96 MASSACHUSETTS Institute of Technology (MIT) 12 COLUMBIA University 21 MCGILL University Canada 13 DUKE University 98 14 26 University of PENNSYLVANIA 97 15 23 JOHNS HOPKINS University 16 AUSTRALIAN National University Australia 17 19= University of TOKYO Japan 18 33= University of HONG KONG Hong Kong 95 19 STANFORD University 20= 35= CARNEGIE MELLON University

European Results COUNTRY 2006 RANK 2005 RANK NAME PEER REVIEW
RECRUITER REVIEW INT'L FACULTY INT'L STUDENTS FACULTY/STUDENT CITATIONS/FACULTY OVERALL 2= 2 University of CAMBRIDGE United Kingdom 100 99 83 98 91 97.6 3 University of OXFORD 82 97 96 5 9 Imperial College LONDON 81 97.5 25 UCL (University College LONDON) 95.3 23 33= University of EDINBURGH 76 71 80 88.8 24 46= King's College LONDON 90 95 70 93 84 88.2 26 18 Ecole Normale Supérieure, PARIS France 60 61 87.1 28 37 ÉCOLE POLYTECHNIQUE 94 78 85.1 30 40 University of MANCHESTER 88 77 85 84.7 64= University of BRISTOL 72 84.1 42 ETH Zurich (Swiss Federal Institute of Technology) Switzerland 92 75 74 82.5 48 69 University of AMSTERDAM Netherlands 32 78.6 53= TRINITY College Dublin Ireland 58 76.9 57 73 University of WARWICK 62 89 76.4 59 17 LONDON School of Economics and Political Science 65 29 75.7 58= HEIDELBERG University Germany 63 87 75.5 Katholieke Universiteit LEUVEN Belgium 39 51 55 75.0 86 DELFT University of Technology 66 67 74.4 65= 90= University of BIRMINGHAM 74.1 Ludwig-Maximilians-Universität München (MUNICH)

European Results COUNTRY 67 82= Technische Universität MÜNCHEN Germany
2006 RANK 2005 RANK NAME COUNTRY PEER REVIEW RECRUITER REVIEW INT'L FACULTY INT'L STUDENTS FACULTY/STUDENT CITATIONS/FACULTY OVERALL 67 82= Technische Universität MÜNCHEN Germany 68 71 88 69 59 83 73.9 102= University of SHEFFIELD United Kingdom 96 81 73.7 70 85 University of NOTTINGHAM 98 64 65 84 73.2 71= 111= UPPSALA University Sweden 34 72 51 26 73.0 74= 124= University of YORK 62 91 77 76 72.4 109= University of ST ANDREWS 57 95 78 99 72.3 80= 121 University of LEEDS 74 97 56 72.0 141= University of SOUTHAMPTON 60 90 University of GLASGOW 75 42 54 71.8 90= LEIDEN University Netherlands 63 35 93 40 71.7 85= 87 University of VIENNA Austria 86 80 12 89 71.6 UTRECHT University 55 38 24 70.9 93= 54= University of COPENHAGEN Denmark 82 66 70.1 CARDIFF University 68.6 100 116= University of HELSINKI Finland 79 45 28 68.2 101 139 University of LIVERPOOL 68.1 105 39 University of GENEVA Switzerland 67.2 106 122 LUND University 41 43 52 66.9 109 132 DURHAM University 49 92 61 66.5 111 172= MAASTRICHT University 66.2

Europe vs. North America

Faculty Level Data Peer opinion Citations per paper Not aggregated
No citations and arts and hums Not citations per person as no staff numbers Invalid to aggregate two very different measures Cf Chris Hood More on this later

Faculty Level: Top 5s Arts & Humanities Engineering & IT
Harvard Berkeley Oxford Cambridge Yale Engineering & IT MIT Stanford Caltech Life Sciences & Biomedicine Johns Hopkins Natural Sciences Berkeley MIT Cambridge Harvard Princeton Social Sciences LSE Yale Stanford

Faculty Level: Europe Top 5s
Arts & Humanities Oxford Cambridge LSE UCL Edinburgh Engineering & IT Imperial ETH Zurich TU Delft Life Sciences & Biomedicine Karolinska Natural Sciences Cambridge Oxford Imperial ETH Zurich ENS Paris Social Sciences LSE UCL Warwick

The Results Available at www.topuniversities.com
Top 100 for the following subject areas Arts & Humanities Engineering & IT Life Sciences & BioMedicine Natural Sciences Social Sciences World’s oldest universities Year on year results

THE RESPONSE

Response More work than writing the thing
In 2006 about 30 newspaper articles in Mexico alone Interest from media, universities etc across Europe and Asia Significant influence on web traffic Less, but growing, interest from the US Esp Switzerland, Malaysia, Ireland, Southern Europe, Japan, say a little on each

Types of response Who told you that? Reject the whole idea
Complain about their position Think it is about right Wonder how to do better Very transparent Well, here goes….

THE FUTURE

Online Database Submit a broad range of quantitative and qualitative data (including those we need for the rankings) about your institution using our online interface Populate and enrich your profile on Educate visitors on other aspects of your institution

Data Validation Hierarchy
Central Statistics Agency/Department e.g. HESA (UK), NCES (US) Direct contact with institution Data available on institution’s own website Data from previous year If sourced via method 1 or 2 Third party data e.g. World of Learning “Smart” Average National or regional, may exclude outliers Scaled based on available data

Recruiter Review Supply us with lists of employer contacts for the recruiter review Exclusive single use only Benefit your institution in recruiter review Improve the overall strength of recruiter review

Broadening Engagement
2007 Korea, Japan, Thailand, Philippines, Netherlands, Hong Kong, India, Taiwan, Malaysia, Singapore, Australia, New Zealand, Korea, Malaysia, Iran, US (NAFSA), UK, Hong Kong (APPLE), China 2008 (planned to date) UK, Ireland, Japan, Singapore, Malaysia, Thailand, Taiwan, Korea, US, Australia, Belgium

Coming Soon! Recruiter Review Breakdown
Preview Access to Recruiter Survey Academic Sign Up Facility Subject Focuses Personalised Rankings Innovation Showcase Engagement Database Entry Requirements Translator PhD Matching Engine

Ben Sowter ben@qsnetwork.com
Thank You Ben Sowter

Ben Sowter Head of Research QS

Similar presentations

Presentation on theme: "Ben Sowter Head of Research QS"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Ben Sowter Head of Research QS

Similar presentations

Presentation on theme: "Ben Sowter Head of Research QS"— Presentation transcript:

Similar presentations

About project

Feedback