1. Political Context of Research Evaluation Luke Georghiou

2. Globalisation of science and industry Growing scientific collaboration and comparison Linkages to firms no longer national and rationale for science as attractor of investment Evaluation following internationalisation Comparative studies and benchmarking studies drawing on evaluation Shift from evaluation of returns to evaluation of capability of a national science base

3. Interdisciplinarity & peer review Interdisciplinarity growing in scale and scope - “massive interdisciplinarity” Clash with disciplinary basis of peer review eg in UK’s Research Assessment Exercise What is the alternative for peer review? Also challenge to “gold standard” of refereed publications from new types of publication behaviour

4. Increased emphasis on evidence of economic exploitation of research Studies showing general case strong but demonstration of effects at programme level problematic New institutional pathways to exploitation imply different ways of measuring the economic contribution of research –eg sale of equity in start–ups by universities –Contribution of R&D to the service sector

5. Increased emphasis on evidence for achievement of social goals through research Social goals equate to environment, health, safety plus societal goals such as reduced unemployment and social exclusion Consequences of extending the stakeholder group to consumers of public goods

6. New rationales for innovation policy & new instruments Systemic rather than market failure rationale for intervention focuses on bridging institutions Evaluation looking at bridging institutions changes focus –Detection of networks requires sensitive research instruments –Lack of substantial government financial input means light touch and persuasion needed to secure cooperation of actors with evaluation –Difficulty of reconciling with external government requirements designed for traditional policies

7. Institutional reform Reform of government labs led to contractualisation, agencification and privatisation Loss of legitimacy for institutional evaluation - leaves programme evaluation as main tool Potential loss of legitimacy in university sector as government diminishes in importance as a funder Relation between performance indicators and evaluation –fundamental requirement for understanding of context, goals and relationships which link goals to effects

8. The customers’ challenge Increasing tendency for customers to specify in detail (over-specify?) how an evaluation should be performed Loss of scope for creative development Specification may be done by those less expert than the evaluators Opens opportunity for consultants to win contracts for routine evaluations with minimal critical content

9. The auditors’ challenge Research & innovation policy exceptional in being serviced by specialist evaluation community Auditors emerging from traditional territory on the back of indicators movement Dangers of wrong indicators & out of context use Research evaluation community must demonstrate that it offers more credible & accurate alternative