Presentation is loading. Please wait.

Presentation is loading. Please wait.

Real world approaches to assessing the impact of research on policy Annette Boaz Kings College London 23 rd September 2010.

Similar presentations


Presentation on theme: "Real world approaches to assessing the impact of research on policy Annette Boaz Kings College London 23 rd September 2010."— Presentation transcript:

1

2 Real world approaches to assessing the impact of research on policy Annette Boaz Kings College London 23 rd September 2010

3 2 Outline Models of the relationship between research policy and practice Rationales for assessing impact What can we learn from the literature? 10 research impact assessment case studies Consider the implications for SSH research

4 3 Models Linear – relay race Relationship – two communities Systems – co- production Diffusion Engineering

5 4 Activities designed to promote research impact Knowledge brokers Research briefings Participation in Networks Co-locating researchers and policy makers Joint research projects Advisory groups Seminars/ presentations

6 5 Models Linear – relay race Relationship – two communities Systems – co- production Research briefings Joint research projects Advisory groups

7 Effective knowledge transfer depends on the nature of the research and its utility the personal characteristics of both researchers and users the links between researchers and users the context for the application of the research See Nutley, Walter and Davies, 2007, pp 81-83, See also Overseas Development Institute 6

8 7 Why assess the impact of our research? Learning Performance In its quest for excellence, the European Commission must and will encourage, promote and support every effort to understand and monitor the quality of research at universities. 1 Accountability Promoting organizational achievements Moderating between stakeholders 1 Commissioner Janez Potočnik, Assessing Europes University-Based Research, 2010

9 8 Review question: How do you (best) evaluate the impact of research on policy? Explored the websites of 30 organizations Hand search of selected journals and reference lists Search of SPRU and IDS library catalogues Search of 10 research databases UK and international sources

10 Research programmes Research Initiatives to increase research use Research Users Systematic reviews Focusing the review on policy use Single study Policy makers

11 Other includes: Brazil, Denmark, Spain and Norway

12 11 Methods for measuring research impact

13 How do people evaluate the impact of research programmes? Wide range of methods and approaches (often in combination): · Qualitative methods: semi-structured interviews, documentary analysis, field visits and observations · Quantitative methods: surveys, bibliometrics and patent/new technology tracking · Panels and peer review · Workshops and focus groups · Process tracking

14

15 Ten impact assessment case studies Stockholm Environment Institute 14 The Foundation for Research, Science and Technology (New Zealand) Finnish Environment Institute The Irish Environmental Protection Agency CGIAR Environment Agency (England and Wales) Defra Land and Water Australia French Ministry of Ecology, Energy, Sustainable Development and the Sea Swedish Environmental Protection Agency

16 15 The Irish Environmental Protection Agency

17 Finnish Environmental Institute Embedded approach: Ten year independent evaluation, four year strategic evaluations, internal annual evaluations, annual researcher self evaluation Goal: how to increase impact Methods: multi-method 16

18 Key themes: Diverse of objectives, scope and scale Types of approach Generic challenges Challenges specific to environmental research 17

19 What are the implications for SSH research in Latin America and Europe? 18

20 8 key questions (when designing an impact evaluation) 1. What is your conceptual framework? 2. What are the outcomes of interest? 3. What methods will best explore the outcomes of interest? 4. How do you address attribution? 5. What is the direction of travel for the evaluation? 6. Is this a mixed method approach, providing scope for triangulation? 7. Will the methods selected capture context and the complexity? 8. When might be the best time to conduct the evaluation? These need to be considered in the light of the time, skills and resources available for the evaluation. See Boaz et al 2009

21 My contact details: Some useful references and websites: -Boaz, A, Fitzpatrick, S and Shaw, B (2009) Assessing the impact of research on policy: a literature review. Science and Public Policy, 36(4): Nutley S, Walter I, and Davies H (2007) Using evidence: How research can inform public services. Bristol: The Policy Press. -OLeary, G, Boyle, R, Donlon, B and Sheils, L (2008) Providing a framework for accountability and learning in environmental research. Administration. 5 (4): The Overseas Development Institute RAPID programme: - Wooding S, Nason E, Klautzer L, Rubin J, Hanney S and Grant J (2007) Policy and practice impacts of research funded by the ESRC. A case study of the Future of Work programme, approach and analysis. Cambridge: RAND Europe. 20

22 21 Managing Editors: Annette Boaz, Division of Health and Social Care Research, King's College London, UK and David Gough, Institute of Education, University of London, UK Volume 6, 2010, 4 issues: January, May, August and November


Download ppt "Real world approaches to assessing the impact of research on policy Annette Boaz Kings College London 23 rd September 2010."

Similar presentations


Ads by Google