Mauro Degli Esposti Towards a new model for evaluating the merit of Italian academic scientists and their institutions Ferrara, Jan 2011

A new analysis of Italian universities – the start From The Times October 9, 2008 Whatever happened to the University of Bologna? The University of Bologna, which received its charter from Frederick I Barbarossa in 1158, was probably the first university founded in the Western world. It became a renowned home to scholars of great distinction in both the sciences and the humanities. The Times Higher Education-QS World University Rankings survey was published this week. It rates the performance of universities according to the quality of their research, the employment rates of graduates and their international profile. The University of Bologna does not feature in the top 100! Manchester = 29 – now 26 Harvard = 1, Cambdrige = 3, Oxford = 4, UCL = 7... M Degli Esposti, presented at the first conference of Via-academy, 21 September

MERIT – a new approach to evaluation M Degli Esposti, presented at the first conference of Via-academy, 21 September The members of the Via-academy network who work in UK have been selected on the basis of merit, but educated in Italy. The number of academic links of Via-members with Italian institution will be proportional to the relative teaching capacity of these institutions – i.e. they will be an output measure of their quality and relative standing. Diritto di cronaca 12/10/2009 Un ateneo per scappare di FLAVIA AMABILE

Italian University Times QS Leiden 250VIA members Bologna 1 33 Roma 1 La Sapienza 2 22 Padova 4 44 Pisa 5 76 Firenze 6 88 Napoli Federico II 7 55 Milano Kendall tau correlation coefficient 0.31 with Leiden 1 With Leiden 0.79** with Leiden by Marco Geraci M Degli Esposti, presented at the first conference of Via-academy, 21 September 2009

by Marco Geraci Italian University VIA25Gelmini virtuose (27+) Padova114 Bologna216 Milano311 Roma 1 La Sapienza4(42) Torino58 Pisa622 Firenze727 (later) Pavia825 Trento91 Napoli Federico II10(38) Kendall tau correlation coefficient 0.63** with Leiden 0.09 with Leiden VIA25 - cumulative ranking from 10 separate rankings October 2009

M Degli Esposti, presented at the second conference of Via- academy, 18 September 2010

From top Italian scientists: ranking of Italian institutes La Stampa 30/8/10 M Degli Esposti, December 31, 2010 – retrieved from: s_VIA-Academy.aspx 400 TIS 1230 TIS La Repubblica 5/1/2011

The media resonance of top Italian scientists

Italian biochemists and their h-index Note Valori di h-index sono stati calcolati cosi: X, con Google Scholar add-standard, con iniziale, ristretto alle aree chemistry, biology and medicine Y, con ISI Web of Science, In genere all years nei settori di bio-medicina (non clinici) e chimica (e.g. medicinal chemistry) Mauro Degli Esposti La Voce.it, mean for full professors = 19

Italian scientists: home and abroad Abroad (50) Home - in Italy (50) >70 TIS Reports 001

Top Italian Scientists – towards the saturation of names? 83% of the top100 work inBio-medicine! M Degli Esposti, December 31, 2010 – retrieved from: op_italian_scientists_VIA- Academy.aspx

Top Italian scientists abroad (total = 363) USA UK Swizerland France Germany Sweden Spain Austria Vatican Canada Belgium Other nations number M Degli Esposti (2010), retrievedJanuary 3, 2011 from:

Fonte : OCSE Nel 2006 Italia = ricercatori

Areas of relative excellence for Italy top scienziati per discipline medicina biologia fisica astrofisica chimica informatica farmacologia economia matematica technology numero di scienziati M Degli Esposti, December 31, 2010 – retrieved from: op_italian_scientists_VIA- Academy.aspx

Fonte : ISTAT TIS

Ranking di universita statali e non – problems Luniversità italiana ha finito anche la frutta di Andrea Aparo IL Fatto Quotidiano, 12 Novembre 2010 Ricercatori italiani uguale fannulloni? di Pietro Greco Scienza in Rete ( 04/05/2009

Ranking for Italian universities – various and different Marco Geraci e Mauro Degli Esposti, unpublished results, October 2010 RankingFrom sourcenature Times QS - THEUKQS Agency and TimesMix of 6 parameters ARWUChinaJiao Tong Shanghai UniversityMix of 6 parameters (different) Leiden top 250HollandCSTS Leiden UniversityBibliometric (normalized) WebometricsSpainCybermetrics lab, CISCBased on data of web visibility VisionUK - ItalyVision think-tankMix of 5 parameters (2009) Campus superItalyCampus magazineAverage of 8 rankings Sole 24 OreItalySole 24 Ore newspaperMix of national parameters Repubblica CensisItalyCensis InstituteMix of national parameters MIURItalyCIRV, VTR Bibliometric (selected)

Marco Geraci e Mauro Degli Esposti, November 2010, in preparation In depth analysis of all ranking – recent advances GRAND VIA - a new meta-ranking combining 5 international and 4 national ones From co-variance patterns – principal components analysis, PCA (Pearson, 1901)

Ranking of Italian universities: 2 principal components Principal component analysis of different rankings of Italian universities International impact and visibility National performance Top Italian Scientists Vision Marco Geraci e Mauro Degli Esposti, November 2010 in preparation Grand VIA

Ranking from top Italian scientists – better than others! Marco Geraci e Mauro Degli Esposti, November 2010 Computed over 4 different international rankings Vision

Evaluation landscape of state Italian universities Marco Geraci e Mauro Degli Esposti, submitted

Landscape of state Italian universities - all Marco Geraci e Mauro Degli Esposti 2010, submitted… Ferrara

Ferrara university – ranking among top institutes in Italy: before new TIS = 26Total TIS = 15 and after systematic analysis 31-12

Ferrara university – ranking among state universities: before Total TIS = 15 Ranking = 15 M Degli Esposti, December 15, 2010 – retrieved from: IA-Academy.aspx + 12 new TIS = 27 and after systematic analysis Ranking = 10 M Degli Esposti, January 5, retrieved from: IA-Academy.aspx

Different Area – different impact (publications and citations) Normalization? By which criteria? Which database to use for reference? M Degli Esposti, September, 2010 – retrieved from: Manueletto del Ricercatore Provetto NOT ISI WoS!

Scientometrics 73 (2007) The normalization factor follows a stretched exponential; it becomes bigger by increasing the number of published articles For scientists with less than 50 published articles, the field- specific differences are small

Note Valori di h-index sono stati calcolati cosi: X, con Google Scholar add-standard, con iniziale, ristretto allarea egineering and mathematics Y, con ISI Web of Science, all years, ristretto ai settori matematici Matematici con nomi Compositi (e.g. De Giorgi) sono esclusi - ISI non li trova! Mauro Degli Esposti – Via-academy.org, h-index - discrepancies between different databases NOTA: usando Google Scholar si normalizza gia le citazioni dei matematici rispetto a quelle che si computano con ISI WoS. Questo risulta in una normalizzazione automatica di un fattore = 1.67, Simile al fattore di nromalizzatione proposto da Iglesias (2007) ca. = 1.83

Discrepancies between different databases for h-index: the problem with chemistry Add-on PorP ISI WoS Average h-index Google Scholar database ISI All chemists of Ferrara university Mauro Degli Esposti, Correlation factor = 1.38

Average h-index Total citationsNumber of TIS Mauro Degli Esposti, average h-index Ferrara departments y = x R 2 = PorP (initial) Add-On (full name)

Fonte : National Science Board

Biomedicine: area variation of publication rate Publications per year x 10-3 Relative disease burden (Log %) Correlation, R 2 = 0.72 Neoplasms %articles that are patent-associated[

Ordine alfabetico

Fonte : OCSE

MERIT -evaluation MERIT - evaluation Lets the discussion begin!

Fondazione CRUI luglio 2010 CAF = Common Assessment Framework H-index, pagina 65 (su 85)

Evaluate university institutions and individuals: The VIA-Ferrara model A 3-step new evaluation model Mauro Degli Esposti, 6 December 2010 TIME FRAME: end of 2010, beginning of 2011

Evaluate university institutions and individuals: The Manchester model Mauro Degli Esposti, 6 December 2010 Total weight Publications = 40% of RQI

Evaluate university institutions and individuals: The Manchester model Mauro Degli Esposti, 6 December 2010 Before RAE 2009: Category - weight Esteems5 Publications75 Environment*25 PROBLEMS: 1. Evaluation of Publication quality by Internal Committee – prone to politics! 2. No consideration of international standing and individuals scientific impact, e.g. h-index *including research income Research Quality Indicator = RQI

Evaluate university institutions and individuals: The VIA-Ferrara model A 3-step new evaluation model 1. Via-academy Quantitative evaluation of the scientific impact of all Ferrara scientists based on the systematic analysis of h-index. Area-specific analysis of h-index using average values of top Italian scientists as reference 2. Ferrara All scientists to provide by a list with their top 4 papers in the last 2 years; the number of Ph.D. and Master students supervised; the total grant income obtained from external sources. TIME FRAME: end of Combine the analysis 1 with data in 2 to define the basic format (like in Manchester) and quantify individual profiles - with 1 at least counting for 50% Mauro Degli Esposti, 6 December 2010

Questions? Simple model for auto-evaluation in specific disciplines within a university Set a threshold of intrinsic (local) excellence by applying this formula: Threshold = average + SD + 1 (in absolute value of h-index) For example, Ferrara chemists: Threshold = = 24

Italian scientists: home and away Nota: la barra corrisponde alla deviazione standard. Media dei valori di h-index per gruppi di scienziati ed accademici italiani che lavorano in Italia. I valori sono simili a quelli degli Italiani che lavorano allestero. Mauro Degli Esposti – La Voce.it, Valore soglia per entrare in lista

Su un totale di 1188 scienziati nella TIS al , Di cui 834 in Italia – escluso CNR ed INAF M Degli Esposti, December 15, 2010 – retrieved from: