Philip Davies Using Evidence for Policy and Practice Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making Capacity Building Workshop 1st and 2nd June 2015 Pretoria, South Africa

Philip Davies Instrumental Use Involves acting on research results in specific, direct ways. Conceptual Use Involves using research results for general enlightenment; results influence actions, but in less specific, more indirect ways than in instrumental use Symbolic Use Involves using research results to legitimate and sustain pre-determined positions. How Evidence is Used in Policy Making

Philip Davies Instrumental, conceptual and symbolic uses of evidence are not mutually exclusive, but can operate in different ways at different stages in the policy cycle and under different political contexts. How Evidence is Used in Policy Making

Philip Davies Tested four variants of a means-tested conditional cash transfer paid to 16–18-year olds for staying in full- time education. Two levels of payment (£30 and £40) to either the young person or a primary carer (usually the mother); Combined with different levels of a retention bonus (£50 and £80) and an achievement bonus (£50 and £140). Male and female young people, and in both urban and rural areas. Comparison groups using propensity score matching The Educational Maintenance Allowance Instrumental Use of Evidence

Philip Davies Case Study - The Employment Retention and Advancement (ERA) Demonstration The Policy Issue: How to retain, and advance, low-pay/no-pay workers in the labour market Evidence Gathering: Systematic review of the global evidence on labour market and social welfare participation Secondary analysis of existing primary data (surveys and admin) Stakeholder analysis and involvement Empirical testing in six different labour market areas using experimental design (randomised controlled trial)

Philip Davies Case Study - The Employment Retention and Advancement (ERA) Demonstration Tested the likely impact and cost-effectiveness of:  A post-employment adviser service,  Cash rewards for staying in work and for completing training  In-work training support Random assignment of over 16,000 people from six regions of Britain: ERA programme or a business-as- usual control group. Seven years to complete – a strategic development evaluation But milestone data in real time from Year 1

Philip Davies [Extreme] Summary of Outcomes The ERA Demonstration Project ERA evaluation had a differential effect on different types of low-pay/no-pay workers [Anticipate heterogeneity of impacts] Initial impacts in Year1 were not sustained Initial small impacts in Year 1 had larger impacts in Years 3 and beyond The timing of impact measurement/monitoring is important Requires regular built-in M&E

Philip Davies Barriers to the Use of Evidence Policymakers’ lack of familiarity with the research process Researchers’ lack of familiarity with the policy process Trust (lack of trust) of policymakers in researchers (vice versa) Timeliness and availability of evidence Physical access to evidence Cognitive access to evidence ( i.e. lack of understanding) Lack of clarity in the presentation of evidence

Philip Davies Interactions between researchers and policymakers increases the prospects for research use by policymakers (Lavis et al, 2005). Early and ongoing involvement of relevant decision makers increase research utilisation (Lomas, 2000) Identify and use interpersonal networks and interactions Identify willing and able knowledge brokers Separate strategic from operational demands for evidence Get policy makers to own the evidence – not just the policy Overcoming Barriers to the Use of Evidence

Philip Davies Establish what research says and does not say Establish the policy messages and policy implications Use a 1:3:25 format Be clear - plain English summary Be persistent and opportunistic Improving Communication of Evidence

Philip Davies The ‘One’ in the 1:3:25 Format A one page of main messages The lessons decision makers can take from the research Not a summary of findings Implications of findings No details of methodology

Philip Davies These are the key findings of the study The classic Executive Summary Condensed to serve the needs of the busy decision maker Focus on how the study will be useful for policy Some brief mention of methodology Some implications of policy and practice The ‘Three’ in the 1:3:25 Format

Philip Davies The ‘25’ in the 1:3:25 Format This should include: Context/Background Approach (Methodology in Appendices, not text) Results Implications Knowledge gaps References Additional resources Appendices

Philip Davies Group Exercise Identify networks of policy makers and evidence providers in your work environment who might help to overcome barriers to using evidence Consider what actions might need to be taken in your work environment to get evidence into policy/practice Consider how the capacity to find, appraise and use evidence might be developed in your work environment

Philip Davies Thank you Philip Davies (0) Visit