1. www.3ieimpact.org Philip Davies International Initiative for Impact Evaluation [3ie] Getting Evidence Into Policy 3ie-LIDC Seminar Series 'What Works In International Development’ London School of Hygiene and Tropical Medicine 20 th February 2013

2. www.3ieimpact.org Philip Davies Outline of Presentation How evidence is used in policy Factors other than evidence Some features of evidence Different notions of evidence Barriers to getting evidence into policy Overcoming barriers to getting evidence into policy

3. www.3ieimpact.org Philip Davies Instrumental Use Involves acting on research results in specific, direct ways. Conceptual Use Involves using research results for general enlightenment; results influence actions, but in less specific, more indirect ways than in instrumental use Symbolic Use Involves using research results to legitimate and sustain pre-determined positions. How Evidence is Used in Policy Making Source: Carol Weiss, 1982

4. www.3ieimpact.org Philip Davies “Rarely does research supply an “answer” that policy actors employ to solve a policy problem. Rather, research provides a background of data, empirical generalisations, and ideas that affect the way that policy makers think about a problem.” “But to acknowledge this is not the same as saying that research findings have little influence on policy.” How Evidence is Used in Policy Making Source: Carol Weiss, 1982

5. www.3ieimpact.org Philip Davies Factors Other Than Evidence Evidence Experience & Expertise Judgement Resources Values and Decision Making Context Bureaucratic Culture Lobbyists & Pressure Groups Pragmatics & Contingencies

6. www.3ieimpact.org Philip Davies Evidence is almost always probabilistic Evidence is often context specific Often disagreement agreement on what counts as ‘evidence’ Evidence is always contestable/contested Evidence is rarely self-evident Not all research is of equal value/sufficient quality Single studies can misrepresent the balance of evidence Hence, the need for systematic reviews/synthesis of evidence Some Features of Evidence

7. www.3ieimpact.org Philip Davies Different Notions of Evidence Policy Makers’ Evidence Colloquial (Narrative) Anything that seems reasonable Policy relevant Timely Clear Message Researchers’ Evidence ‘Scientific’ (Generalisable) Proven empirically Theoretically driven As long as it takes Caveats and qualifications Source: J. Lomas et al, 2005 Knowledge Translation and Transfer

8. www.3ieimpact.org Philip Davies UK Policymakers’ Views of Evidence* Focus on the ‘end product’, rather than how the information was either collected or analysed Use of ‘anecdotal’ evidence Drawing on such things as ‘real life stories’, ‘fingers in the wind’, ‘local’ and ‘bottom-up’ evidence But: “If we try and move anywhere without having the scientific basis to do so we get fleeced in the House” *Source. Campbell, S. et al, 2007: Analysis for Policy, London, GSR.

9. www.3ieimpact.org Philip Davies Sharks Where Do UK Civil Servants Go For Evidence? Plankton Academic/Evaluation Research?

10. www.3ieimpact.org Philip Davies Barriers to the Use of Evidence Policymakers’ lack of familiarity with the research process Researchers’ lack of familiarity with the policy process Trust (lack of trust) of policymakers in researchers (vice versa) Physical access to evidence Cognitive access to evidence (i.e. lack of understanding) Lack of clarity in the presentation of evidence Timeliness and availability of evidence Sources: Lomas, 2000; Petticrew et al, 2004; Lavis et al, 2005; Dobbins et al, 2007; Ouimet, et al, 2009; Rosenbaum, 2010

11. www.3ieimpact.org Philip Davies Overcoming Barriers to the Use of Evidence Early and ongoing involvement of relevant decision makers Interactions between researchers and policymakers increases the prospects for research use by policymakers Identify and use interpersonal networks and face-to-face interactions Identify willing and able knowledge brokers Separate strategic from operational demands for evidence Get policy makers to own the evidence – not just the policy Sources: Lomas, 2000; Petticrew et al, 2004; Lavis et al, 2005; Dobbins et al, 2007; Ouimet, et al, 2009; Rosenbaum, 2010

12. www.3ieimpact.org Philip Davies Establish what the evidence says, and does not say Establish the policy messages and policy implications But avoid ‘recommendations’ Provide information about the costs and benefits, harms and risks of interventions/policies Present contextual factors that affect local applicability Be clear - plain English summary, unclouded by jargon Find the ‘influencing moment’ Aligned to decision making timescales Use a 1:3:25 format Improving Communication of Evidence

13. www.3ieimpact.org Philip Davies The ‘One’ in the 1:3:25 Format A one page of main message bullets The lessons decision makers can take from the research Not a summary of findings Suggest implications of findings No details of methodology

14. www.3ieimpact.org Philip Davies These are the key findings of the study The classic Executive Summary Condensed to serve the needs of the busy decision maker Focus on how the study may be useful for policy Some brief mention of methodology Some implications of policy and practice The ‘Three’ in the 1:3:25 Format

15. www.3ieimpact.org Philip Davies The ‘25’ in the 1:3:25 Format This should include: Context/Background Approach (Methodology in Appendices, not text) Results Implications Knowledge gaps References Additional resources Appendices

16. www.3ieimpact.org Philip Davies Summary The routes of evidence to policy are usually indirect and delayed There are factors other than evidence Evidence is seldom definitive or invariant Nor is it self-evident There are different notions of evidence Physical and cognitive access to evidence is important Evidence has to be ‘worked’ into policy/practice Clear communication evidence is essential

17. www.3ieimpact.org Philip Davies Campbell, S., et al, 2007 Analysis for Policy, London, GSR Dobbins et al, 2007 Public Health Decision-Makers’ Informational Needs and Preferences for Receiving Research Evidence, Worldviews on Evidence-Based Nursing, 3 rd Quarter, 156-163, New York, Wiley Inc. Lavis et al, 2005 Towards systematic reviews that inform health care management and policy-making, Journal of Health Services Research and Policy, 10, Suppl 1 July 2005 S1:35. Lomas, J., 2000 Connecting Research and Policy, Canadian Journal of Policy Research, Spring, 140- 144 Lomas, J. Culyer, T., McCutcheon, C., McAuley, L., and Law, S., 2005 Conceptualizing and Combining Evidence for Health System Guidance, Final Report, Canadian Health Services Research Foundation, Ottawa.. References

18. www.3ieimpact.org Philip Davies Ouimet, M., Landry, R., Ziam, S., and Bédard, P., 2009 The absorption of research knowledge by public servants, Evidence and Policy, 5, 4, 331-350. Petticrew, M., et al, 2004 Evidence for public health policy on inequalities: 1: The reality according to policymakers, Journal of Epidemiology and Community Health 2004;58:811-816 Rosenbaum, S.E., et al, 2010 Evidence summaries tailored to health policy-makers in low- and middle- income countries, Bulletin of the World Health Organization, 89, 1, 54-61. Weiss, C.A., 1982 Policy research in the context of diffuse decision making, Journal of Higher Education, 53, 6, 619-639. References (continued)

19. www.3ieimpact.org Philip Davies Thank you Philip Davies Email: pdavies@3ieimpact.orgpdavies@3ieimpact.org +44 (0)207 958 8350 Visit www.3ieimpact.org