1. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Positional impact of reputation in global university rankings Philippe Vidal and Ghislaine Filliatreau Observatoire des Sciences et Techniques, Paris

2. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Reputation is an idle and most false imposition, oft got without merit, and lost without deserving Shakespeare, Othello, II, 3

3. IREG Forum University Rankings, Warsaw 16 -17 May 2013 In Global university rankings and their impact. Report II, EUA Report on rankings 2013 2.3 Superficial descriptions of methodology and poor indicators …. Use of poor indicators also persists. In spite of widespread criticisms, reliance on reputation is becoming more and more widespread…..

4. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Rankings graphical comparison as an approach for deconvoluting ranking patterns Reputation surveys: ARWU: 0%, THE: 34.5%, QS: 40% Webometrics taken as a proxy for reputation Facts vs reputation

5. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Methodology 1 Ternary diagrams

6. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Methodology 2 Ranking A vs Ranking B

7. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Methodology 3 Absolute and relative differences Diagrams (ARWU-Y) vs ARWU and (ARWU-Y)/ARWU vs ARWU

8. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Methodology 4 Diagram ARWU/X vs ARWU

9. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Methodology 5 Rankings reproducibilities - years selected: 2010 - 2011 - 2012 - average deviations to the average from one year to next: ARWU: 2-3 % THE: 10-12 % QS: 6-7 % Webometrics: 25-30 %

10. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Methodology 6 Filtering rules 90 % of the universities (259) selected 62 % ranked in 4 rankings 6 % in 3 rankings 22 % in 2 rankings First 250 ARWU ranks: 131 universities

11. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Targets United States United Kingdom Northern Europe (Germany, Scandinavia, Netherlands) Southern Europe (France, Spain, Portugal, Italy)

12. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 1 Ternary diagrams

13. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 2 Ternary diagrams left: ARWU 1->500, right:1->250

14. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 3 ARWU vs THE 1

15. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 4 ARWU vs THE 2

16. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 5 ARWU vs QS 1

17. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 6 ARWU vs QS 2

18. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 7 ARWU vs Webometrics 1

19. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 8 ARWU vs Webometrics 2

20. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 9 US: Relative differences ordinate scale: 1 = 100%

21. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Results 10 Europe: Relative differences THE-QS: Continental Europe Webometrics=whole Europe ordinate scale: 1 = 100%

22. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Summary Underestimatation of universities best ranked for research Overestimation of universities less well ranked for research Clear for THE and QS, even clearer for Webometrics And for the best research universities: US: underestimation THE/QS, overestimation Webometrics UK: overestimation THE/QS, underestimation Webometrics N + S Europe: underestimation THE/QS/Webometrics

23. IREG Forum University Rankings, Warsaw 16 -17 May 2013 Conclusions Reputation concept still on its infancy Necessity to do research on the concept as well as its modalities of implementation