Evidence-based policies and indicator systems 2006 Conducting effective research and analysis to support policy delivery. The Green Book: Appraisal and Evaluation in Central Government (HM Treasury) Tony Munton Assistant Director Research, Development and Statistics Home Office

Evidence-based policies and indicator systems stages at which research and analysis should feed into the policy process: At the start… analysis to support a government decision to adopt a new policy, or to initiate, renew, expand or re-orientate programmes or projects, which would result in measurable benefits and/ or costs to the public. This is the appraisal part of the process. In the middle…. Establish appropriate data collection so that as the chosen option(s) is implemented, useful data can be fed into the evaluation. This is the monitoring part of the process And at the finish... to undertake a retrospective analysis of a policy, programme or project at its completion, conclusion or revision. This is the evaluation part of the process.

Evidence-based policies and indicator systems 2006 The ROAMEF model Is there a clearly identified need? What would happen if we don’t intervene? Is there a market failure? What would happen in the absence of intervention? What will happen if we do intervene? What is to be achieved by the intervention? What would success look like? What inputs, outputs and outcomes are related to the intervention (develop a theory of change)? What are the full range of options that might achieve the desired outcomes? What might different options achieve above the current situation? Identification of the “do nothing” option to be used as a baseline to compare the other options; Identifying the additional costs and benefits of each option in comparison to the “do nothing” option; Valuing the additional costs and benefits of each option in comparison to the “do nothing” option; Adjusting for the different timing and incidence of costs and benefits by discounting them to present values; Consideration of any risks and uncertainties; Initial consideration of what information may be needed at the monitoring and evaluation stages (i.e. to make sure the right information is collected). Establish appropriate data collection so that as the chosen option(s) is implemented useful data can be fed into the evaluation stage of the cycle. Monitor delivery of milestones identified in the theory of change. Evaluate the option(s) that was actually implemented. Follows a similar process to the appraisal but uses historic rather than forecast data. As with appraisal, an evaluation will identify and measure the additional costs and benefits of implemented option(s) in comparison to the “do nothing” option. Ensure that the lessons learnt from the ROAMEF cycle are fed back into subsequent proposals.

Evidence-based policies and indicator systems 2006 Practical steps for social research: Some suggestions from the knowledge brokers Think about potential for forming multi- disciplinary teams Use good theory (and theory of change) Go for effective evaluation strategies (RCTs) Think about effective communication of results