Quality Assessment in the Humanities Wim Blockmans NIAS / KNAW Humanities and the ERC Oslo 20 June 2005

Evaluation in the Audit Society Ever growing processes of Evaluation Ever growing processes of Evaluation –Ex ante : ensure risk-taking in funding –Ex post : legitimize research investment Targets Targets –Quality improvement Costs Costs –Proportionality Effects Effects –Implementation ?

Methods Focus on bibliometry Focus on bibliometry Impact measurement as an enterprise Impact measurement as an enterprise –Profitable only for large, capital intensive disciplines integrated on a global scale Hence : heavy bias in AHCI, as shown by ESF-SCH Hence : heavy bias in AHCI, as shown by ESF-SCH Impact vs. quality Impact vs. quality –Mathematician Michael Donaldson –Flaws by cliques –Immediate effects vs. long-term impact Empirical tests of Einsteins theories Empirical tests of Einsteins theories Measuring is only one instrument in evaluation - value Measuring is only one instrument in evaluation - value

Outcomes Administrations seek objective adjudication mechanisms Administrations seek objective adjudication mechanisms Weakness of H&SS performance indicators Weakness of H&SS performance indicators Natural sciences & Medicine score best Natural sciences & Medicine score best –EURYI 2004 : 22 awards vs. 3 for H&SS –NWO-Spinoza : 39 vs. 11 –League of 12 Research Intensive Universities

Effects Fragmentation weak basis Fragmentation weak basis –Mainly individual projects –Few cases of successful interdisciplinary policy-driven programmes Less impulses for large-scale research Less impulses for large-scale research Lack of bibliometric arguments Lack of bibliometric arguments Negative spiral ? Negative spiral ?

Character of Humanities Research Bound to particular society, culture, language Bound to particular society, culture, language Universality of results not evident Universality of results not evident Complexity of explanatory models Complexity of explanatory models Public recognition of relevance Public recognition of relevance Variety of addressees Variety of addressees –Peers –Authorities, professionals –General Public

What to do ? Formulate large programmes Formulate large programmes Capitalize public interest Capitalize public interest Emphasis on multiple functions of Humanities research Emphasis on multiple functions of Humanities research Refuse simple bibliometric methods Refuse simple bibliometric methods Open eye for societal relevance of Humanities researh Open eye for societal relevance of Humanities researh