1. Measuring the Contribution of Sustainable Development Research to Society and Policy-Making Luke Georghiou Manchester Institute of Innovation Research Manchester Business School

2. Some initial trade-offs Research as consumption good vs research as an investment –In first case value is cultural, in second there are users with the expectation of a return at some point in the future Outputs vs Impacts –Output may be scientific publication or even new product or service but impact only realised when output interacts with economy or society eg sales or green technology or application of new regulation

3. More trade-offs Short term vs. long term –many different time-profiles over which impacts are manifested including short-term effects which terminate abruptly as market conditions change outputs which are not used for some years and then become very important, perhaps because complementary technologies have been developed extrapolation is dangerous. Intended vs. unintended –excessive focus on project or programme goals as the basis for evaluation could lead to important unintended effects being missed eg in medical research when a drug developed for one condition turns out to be important in the treatment of another –Unintended effects also include most of those in the negative category.

4. INNOVATION Problems in calculating returns to R&D Timing Attribution Appropriability RESEARCH EFFECTS

5. ICT Technology Pillars Nano-electronics, photonics and integrated micro/nano-system Ubiquitous and unlimited capacity communication networks Embedded systems, computing and control Software, Grids, security and dependability Knowledge, cognitive and learning systems Simulation, visualisation, interaction and mixed realities New perspectives in ICT drawing on other science and technology disciplines Integration of Technologies Personal environment Home environments Robotic systems Intelligent infrastructures Applications Research ICT meeting societal challenges health inclusion mobility environment government ICT for content, creativity and personal development media learning cultural ICT supporting businesses and industry business processes Manufacturing ICT for trust and confidence Future and Emerging Technologies Stronger, lasting growthMore & better jobs Sustainability FP7 A more attractive place to invest and work Internal market Improved regulation Open competitive markets Expand/improve infrastructure Knowledge and innovation for growth Increase R&D Facilitate innovation, uptake of ICT and sustainable use of resources Contribute to strong industrial base Creating more & better jobs More employment & modernise social protection Adaptable workers & flexible labour markets Better education & skills Revised Lisbon Information space Open stable markets for electronic communications & digital services economy Innovation & investment in ICT Deploy services e government Research leadership Investment & improvement Effective adoption of ICT Inclusion & better QoL Knowledge society Social development i2010

6. ICT Technology Pillars Nano-electronics, photonics and integrated micro/nano-system Ubiquitous and unlimited capacity communication networks Embedded systems, computing and control Software, Grids, security and dependability Knowledge, cognitive and learning systems Simulation, visualisation, interaction and mixed realities New perspectives in ICT drawing on other science and technology disciplines Integration of Technologies Personal environment Home environments Robotic systems Intelligent infrastructures Applications Research ICT meeting societal challenges health inclusion mobility environment government ICT for content, creativity and personal development media learning cultural ICT supporting businesses and industry business processes Manufacturing ICT for trust and confidence Future and Emerging Technologies Stronger, lasting growthMore & better jobs Sustainability FP7 A more attractive place to invest and work Internal market Improved regulation Open competitive markets Expand/improve infrastructure Knowledge and innovation for growth Increase R&D Facilitate innovation, uptake of ICT and sustainable use of resources Contribute to strong industrial base Creating more & better jobs More employment & modernise social protection Adaptable workers & flexible labour markets Better education & skills Revised Lisbon Information space Open stable markets for electronic communications & digital services economy Innovation & investment in ICT Deploy services e government Research leadership Investment & improvement Effective adoption of ICT Inclusion & better QoL Knowledge society Social development i2010

7. Overall linkage mapping can become very dense Previous slide shows only a selection of the more obvious linkages While important not to lose the argument of connected rationale must also be able to examine elements of it in isolation to allow more detailed arguments to be developed

8. Different logics Horizontal logic –Exploring interdependency between high level objectives or between intermediate level actions (ie the proposals for FP7) as per previous example Vertical logic –Exploring interdependency between an objective and the relevant parts of FP7 Can isolate as binary link, or Consider combined effect of all aspects of the Programme on that objective, or Consider multiple effects on objectives of a single Programme activity Systemic logic –Considering implications of change across the whole system

9. Vertical logic – binary chain example Pair of binary relationships Overall Lisbon strategy sees increased R&D as necessary condition for growth by making business more innovative, productive etc Activity here is sponsorship of pre- competitive R&D Stated rationale for spending on research rests on: –“European industry lags in investment cf major competitors” –“More intensive cooperation makes most of current capabilities” Growth Increase R&D ICT Technology Pillars

10. Expected impacts and outcomes en route to increased R&D ICT Technology Pillars Immediate impacts Intermediate impacts Ultimate impacts Activities Collaborative projects Networks of excellence Joint technology initiatives etc New consortia formed Companies invest own resources Technology area new to participants More ambitious projects undertaken papers IPR products services New markets entered Increased sales and cost reductions Increased R&D Trained researchers Higher return on R&D Resear chers employ ed

11. Problems continued Timing –Effects may happen long after research –Issues of discounting Attribution –See previous graph – what complementary inputs needed? Linearmodel issues Appropriability –Who gets the benefit from a research project? –Spillovers

12. Inadequacy of linear model of innovation On the one hand we have research which is addressed specifically to solving problems of sustainability –Eg sustainable construction May even be the defining characteristic of a whole field –Eg environmental monitoring On the other hand sustainable solutions may depend upon advances in knowledge in areas of research such as nano, bio, cognitive science and complexity where initial aim was –general advancement of knowledge; or –pursuit of a different objective What kind of translational model is needed?

13. Spillovers Benefits accrue to agents other than the party undertaking the research Rationale for policy intervention social rate of return exceeds inadequate private rate –Eg We may need to subsidise for research leading to green technologies because market alone will not produce them Jaffe describes 3 types –Knowledge spillovers Knowledge created by one agent can be used by another without compensation, or with compensation less than the value of the knowledge Eg reverse engineering, imitation May be deliberate disclosure through publication or patent

14. Spillovers continued Market spillovers –Market forces cause buyers of new product or product made with new process to get some of the benefit because not all superiority of price reduction captured in price (consumer surplus) Network spillovers –Arise when commercial or economic value of a new technology is dependent upon developments in related technologies –Eg communication systems –Firms may fail to coordinate their activities without intervention

15. Nature of socio-economic effects Intermediate outputs Products Processes Services Standards Knowledge and skills Dissemination Impacts/effects Competitiveness Employment Organisation Quality of life Control & care of the environment Regional development Development of infrastructure Production & rational use of energy Industrial development Regulation & policy

16. Nature of Social Impact Often public goods where government is the actual or proxy customer –benefits not captured through market mechanism –paid through taxation –enforced through legislation/regulation Not easily reduced to monetary equivalent –may not be social consensus on valuation of quality of life May be trade-off or complementary relation with wealth creation –co-production of economic and social effects Introduces new stakeholders to evaluation –social groups

17. Science and Quality of Life* * with thanks to Ian Miles

18. Specific Environment Issues Usually link to policy or regulation May involve inter-temporal pricing Some environmental changes are irreversible with implications for future generations May not be obvious who are the “end-users” Problem if proxy-customers not fulfilling duties

19. Impact gap R&DEffect POLICY MEASURE (regulation, law, standard, policy, practice) Evaluation AEvaluation B

20. Practical solutions? UK Government allocates block funding for University research by means of an ex post evaluation every 5-7 years –Previously called Research Assessment Exercise –Allocated funds on basis of quality of outputs (peer review); research environment and esteem) Next exercise renamed Research Excellence Framework

21. REF and IMpact Major innovation to include impact and government insisting that it will count for 25% of score “Impact: An assessment of demonstrable economic and social impacts that have been achieved through activity within the submitted unit that builds on excellent research. This is to assess the extent to which a submitted unit has built upon its strong record of excellent research to make a positive impact on the economy and society within the assessment period. Throughout this document, where we refer to ‘impact’ or ‘social and economic impact’, we include economic, social, public policy, cultural and quality of life impacts. “

22. Method for assessment One case study for every 5-10 people entered Impact must be evident in the 2008-12 assessment period but could come from research up to 15 years before Case study includes –explanation of the nature of the impact; how far-reaching it is/who the beneficiaries are; and how significant the benefits are –appropriate indicators of the impact –an outline of what the underpinning research was, when this was undertaken and by whom –what efforts were made by staff in the unit to exploit or apply the findings or secure the impact through its research expertise –key research outputs that underpin the impact –external reports or documents, or contact details of a user, that could corroborate the impact or the unit’s contribution

23. Indicators for impact on policy Better informed public policy-making or improved public services Research income from government organisations Changes to legislation/regulations/government policy (including references in relevant documents) Changes to public service practices/guidelines (including references in guidelines) Measures of improved public services (for example, increased literary and numeracy rates) Staff exchanges with government organisations Participation on public policy/advisory committees Influence on public policy debate (for example, as indicated by citations by non-government organisations or the media)

24. A framework for impact and internalising spillovers Understanding the relation between research and innovation How to transfer to more sustainable modes of development –overcome lock-in to dominant sociotechnical regimes –Put in place infrastructures and policy/regulatory frameworks Critical role for users and link to demand-side innovation policies Building constituencies including researchers Coordination, engagement and building the capacities and incentives for users to play a major role in the process Embedding in a Grand Challenge initiative more likely to succeed?

25. Research and policy Similar feedback loops to innovation Role of policy users –Risk of purchaser lock-in Is there a need for coordinated Grand Challenge Initiatives to carry forward SD research?

26. Conclusions Linkages to innovation and policy play key role in constructing agenda for SD research Before measuring impact we must have understanding of systemic model which links –Research and innovation –Research and policy The pathways and complementary or intervening factors may not be the same from one sector to another Without evaluation and measurement we work only on theory and belief without the assurance that an evidence base can give us Evidence is needed not only for managers but to reassure the stakeholders who fund and use research