Presentation is loading. Please wait.

Presentation is loading. Please wait.

John Young: ODI, London j.young@odi.org.uk Making Knowledge Count Maximising the value of Research for Development John Young: ODI, London j.young@odi.org.uk.

Similar presentations


Presentation on theme: "John Young: ODI, London j.young@odi.org.uk Making Knowledge Count Maximising the value of Research for Development John Young: ODI, London j.young@odi.org.uk."— Presentation transcript:

1 John Young: ODI, London j.young@odi.org.uk
Making Knowledge Count Maximising the value of Research for Development John Young: ODI, London

2 Overview Part 1: Introduction to ODI & RAPID Definitions
Policy processes & the role of research Policy-makers perspective An Analytical framework Part 2: A Practical framework Some tools for researchers, policy-makers and donors + some examples of their use. Some conclusions Sources of further information.

3 Overseas Development Institute
Development Think Tank 60 researchers Research / Advice / Public Debate Rural / Humanitarian / Poverty & Aid / Economics / Policy Processes DFID, Parliament, WB, EC Civil Society For more information see:

4 RAPID Group Promoting evidence-based development policy & practice
Through Research Advice Public Affairs Capacity-building Working with: researchers policymakers parliamentarians southern think tanks for further information see: /

5 Definitions Research: “any systematic effort to increase the stock of knowledge” Policy: a “purposive course of action followed by an actor or set of actors” Evidence: “the available information supporting or otherwise a belief or proposition” Evidence-based Policy: “public policy informed by rigorously established evidence”.

6 Policy Processes Identify the problem Commission research
Analyse the results Choose the best option Establish the policy Implement the policy Evaluation

7 Monitoring and Evaluation Policy Implementation
Policy Processes Cabinet Donors Monitoring and Evaluation Agenda Setting Decision Making Policy Implementation Policy Formulation Parliament Civil Society Ministries Private Sector

8 Chronic Poverty in Uganda
Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 (http://www.odi.org.uk/publications/working_papers/wp242.pdf)

9 …in reality… “The whole life of policy is a chaos of purposes and accidents. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies 1” “Most policy research on African agriculture is irrelevant to agricultural and overall economic policy in Africa2” “Research is more often regarded as the opposite of action rather than a response to ignorance”3 1 Clay & Schaffer (1984), Room for Manoeuvre; An Exploration of Public Policy in Agricultural and Rural Development, Heineman Educational Books, London 2 Omamo (2003), Policy Research on African Agriculture: Trends, Gaps, and Challenges, International Service for National Agricultural Research (ISNAR) Research Report No 21 3 Surr (2003), DFID Research Review

10 But Policy makers are… …practically incapable of using research-based evidence because of the 5 Ss… Speed Superficiality Spin Secrecy Scientific Ignorance Vincent Cable – Lib. Democrat MP & Shadow Minister of Finance More at:

11 Factors influencing policy making
Experience & Expertise Pragmatics & Contingencies Judgement Evidence Lobbyists & Pressure Groups Resources Habits & Tradition Values and Policy Context Source: Phil Davies Impact to Insight Meeting, ODI, 2005

12 Different Notions of Evidence
‘Scientific’ (Context free) Proven empirically Theoretically driven As long as it takes Caveats and qualifications Researchers’ Evidence Colloquial (Contextual) Anything that seems reasonable Policy relevant Timely Clear Message Policy Makers’ Evidence Source: Phil Davies Impact to Insight Meeting, ODI, 2005

13 Existing theory X Linear model Percolation model, Weiss
Tipping point model, Gladwell ‘Context, evidence, links’ framework, ODI Policy narratives, Roe Systems model (NSI) External forces, Lindquist ‘Room for manoeuvre’, Clay & Schaffer ‘Street level bureaucrats’, Lipsky Policy as social experiments, Rondinelli Policy Streams & Windows, Kingdon Disjointed incrementalism, Lindquist The ‘tipping point’, Gladwell Crisis model, Kuhn ‘Framework of possible thought’, Chomsky Variables for Credibility, Beach The source is as important as content, Gladwell Linear model of communication, Shannon Interactive model, Simple and surprising stories, Communication Theory Provide solutions, Marketing I Find the right packaging, Marketing II Elicit a response, Kottler Translation of technology, Volkow Epistemic communities Policy communities Advocacy coalitions etc, Pross Negotiation through networks, Sebattier Shadow networks, Klickert Chains of accountability, Fine Communication for social change, Rockefeller Wheels and webs, Chapman & Fisher

14 Existing theory – a short list
Policy narratives, Roe Systems of Innovation Model, (NSI) ‘Room for manoeuvre’, Clay & Schaffer ‘Street level bureaucrats’, Lipsky Policy as social experiments, Rondene Policy streams and policy windows, Kingdon Disjointed Incrementalism, Lindblom Social Epidemics, Gladwell The RAPID Framework

15 An Analytical Framework
External Influences Socio-economic and cultural influences, donor policies etc The political context – political and economic structures and processes, culture, institutional pressures, incremental vs radical change etc. The links between policy and research communities – networks, relationships, power, competing discourses, trust, knowledge etc. The evidence – credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of the message, how it is packaged etc

16 Case Studies Detailed: Summary Sustainable Livelihoods
Poverty Reductions Strategy Processes Ethical Principles in Humanitarian Aid Animal Health Care in Kenya Dairy Policy in Kenya Plant Genetic Resources Summary GDN x 50 CSPP x 20 Good news case studies x 5 Mental health in the UK

17 Animal Healthcare in Kenya
1970s 1980s 1990s 2000s Professionalisation of Public Services. Structural Adjustment → collapse of services. Paravet projects emerge. ITDG projects. Privatisation. ITDG Paravet network. Rapid spread in North. KVB letter (January 1998). Multistakeholder WSs → new policies. Still not approved / passed! Professionalisation of Public Services. Structural Adjustment Privatisation ITDG Paravet network and change of DVS. KVB letter (January 1998). Multistakeholder WSs → new policies. International Research ITDG projects – collaborative research. The Hubl Study Dr Kajume Shortly after setting up its first decentralised animal health projects, ITDG organised the first, of what were to become annual “vets workshops” which became the focus of a network of people involved in paravet projects. A conscious effort was made at the start to invite senior government veterinary staff to participate to convince them of the value of the approach. Many NGO and bilateral project staff who were already involved in, or wanted to start decentralised animal health projects, were keen to join the network, and it increasingly focused on practical issues. While this contributed to the rapid spread of the approach across northern Kenya, it neglected to involve senior government policy makers. Dr Kajume, then Provincial Head of Veterinary Services heard about one of these workshops by accident, checked with the Director of Veterinary Services in Nairobi, and was instructed to attend the workshop, tell the participants it was illegal, and close it down. But instead, he became convinced of the value of the approach and persuaded the Director to allow the projects to continue. So paravet projects continued to spread across northern Kenya, deliberately ignored by the Director of Veterinary Services, until the publication of the Kenya Vet Board letter brought matters to a head, and he was forced to do something about it. At which point, Dr Kajume, now Deputy Director of Veterinary Services persuaded him to support the process of multi stakeholder workshops and commission the Hubl study which led to the development of a new policy framework.

18

19 A Practical Framework External Influences political context evidence
Politics and Policymaking Campaigning, Lobbying Policy analysis, & research Media, Advocacy, Networking Scientific information exchange & validation Research, learning & thinking evidence links

20 What you need to know The external environment:
Who are the key actors? What is their agenda? How do they influence the political context? The evidence: Is it there? Is it relevant? Is it practically useful? Are the concepts new? Does it need re-packaging? The political context: Is there political interest in change? Is there room for manoeuvre? How do they perceive the problem? Links: Who are the key actors? Are there existing networks? How best to transfer the information? The media? Campaigns?

21 What you need to do What need to know What need to do How to do it
Political Context: Evidence Links Get to know the policymakers. Identify friends and foes. Prepare for policy opportunities. Look out for policy windows. Work with them – seek commissions Strategic opportunism – prepare for known events + resources for others Who are the policymakers? Is there demand for ideas? What is the policy process? Establish credibility Provide practical solutions Establish legitimacy. Present clear options Use familiar narratives. Build a reputation Action-research Pilot projects to generate legitimacy Good communication What is the current theory? What are the narratives? How divergent is it? Get to know the others Work through existing networks. Build coalitions. Build new policy networks. Build partnerships. Identify key networkers, mavens and salesmen. Use informal contacts Who are the stakeholders? What networks exist? Who are the connectors, mavens and salesmen?

22 Policy entrepreneurs Storytellers Networkers Engineers Fixers
Doing all of these things requires a wide range of skills. Researchers who want to be good policy entrepreneurs also need to be: Storytellers: Practitioners, bureaucrats and policy-makers often articulate and make sense of complex realities through simple stories. Though sometimes profoundly misleading there is no doubt that narratives are incredibly powerful. Networkers: Policy-making usually takes place within communities of people who know each other and interact. If you want to influence policymakers, you need to join their networks. Engineers: There is often a huge gap between what politicians and policy-makers say they are doing and what actually happens on the ground. Researchers need to work not just with the senior level policy-makers, but also with the 'street-level bureaucrats'. Fixers: Policy making is essentially a political process. Although you don’t need to be a Rasputin or Machiavelli, successful policy entrepreneurs need to know how to operate in a political environment - when to make your pitch, to whom and how. Try ODI’s Policy Entrepreneur Questionnaire to find out whether you tend to favour, or to avoid any of these activities. If so, you may need to develop new skills in these areas, or work with others who have these skills.

23 Supply-side Tools Overarching Tools Context Assessment Tools
- The RAPID Framework - Using the Framework - The Entrepreneurship Questionnaire Context Assessment Tools - Stakeholder Analysis - Forcefield Analysis - Writeshops - Policy Mapping - Political Context Mapping Communication Tools - Communications Strategy - SWOT analysis - Message Design - Making use of the media Research Tools - Case Studies - Episode Studies - Surveys - Bibliometric Analysis - Focus Group Discussion Policy Influence Tools - Influence Mapping & Power Mapping - Lobbying and Advocacy - Campaigning: A Simple Guide - Competency self-assessment

24 RAPID Framework

25 Policy Process Mapping
General Context issues – domestic and international. Specific Policy Issues (i.e. the policy cycle) Stakeholder analysis Arena: government, parliament, civil society, judiciary, private sector. Level: local, national, international What is their Interest and Influence? Process matrix + political matrix Political and administrative feasibility assessment [Sources: M. Grindle / J. Court ]

26 Policy Process Mapping
Formulation Implementation Politicians Cabinet Government Bureaucrats Civil Society International Identify key actors that influence SME policy Create Matrix: Organizations and Key Steps of the Policy Process Describe Organizations’ formal position in the policy process Describe Organizations’ informal influence on the policy process Give a number rating (1=low; 5=high) for the influence each organization has on different parts of the policy process.

27 Stakeholder Analysis Why: Steps:
To understand who gains or lose from a policy or project. To help Build Consensus. Steps: Identify Stakeholders Analysis Workshop Develop Strategies Keep Satisfied Engage Closely Monitor (minimum effort) Keep Informed High Power Low Interest

28 Forcefield Analysis Identify what you want to achieve
Identify forces for and against change Identify which are most important Develop strategies to reinforce those for and overcome those against

29 Groundwater in India to maximise impact of DFID forest/ ground water research project in India Researchers, policy makers and activists Used framework to analyse factors in water sector in India Developed strategy for final phase: Less research More communication Developing champions in regional and national government Local, Regional & National advocacy campaign

30 Communications strategy
Identify the audience(s) Identify the message(s) Promotion Evaluate impact and change as necessary Clear Strategy Interactive Multiple formats How? Who? What? Audience Promotion Message In terms of the communications, there are a set of issues which seem to come out most clearly and which make a big difference to whether research is taken up. Communication is crucial in both directions: in terms of researchers listening to policy-makers and in terms of engaging policy-makers in the research right from the beginning and of keeping them involved or in touch with that process. The first step is to identify who you want to influence – the audience. A key aspect of this is to identify what you want them to do differently. Assess their specific information needs, likes and channels (Official / unofficial & Personal / impersonal & empirical data vs stories.) The second step is to clarify your messages – brevity, clarity, what form, language. The third step is promotion – there are many ways, but interactive communications works best; seeing is believing; multiple formats / media are better than one. OSI has been working on Writing Effective Policy Papers – really useful book The policy community & The policy process Structural elements of a paper - Problem description / Policy option / Conclusion Key issues: Problem oriented, targeted, multidisciplinary, applied, clear, jargon-free.

31 Writing Policy Papers Providing a solution to a policy problem
The policy community The policy process Structural elements of a paper Problem description Policy options Conclusion Key issues: Problem oriented, targeted, multidisciplinary, applied, clear, jargon-free. [Source: Young and Quinn, 2002]

32 Organisational development
Knowledge, Attitudes and Practices: The entrepreneurship questionnaire Training & mentoring etc Knowledge Management Organisational development Finance, admin & personnel systems Strategic (action & business) planning Fundraising & reporting Building an organisational profile Communications, Public Affairs and the Media Struyk, 2002, Local Governance Institute, Open Society Network, Budapest

33 Tools for Policymakers
Increasing the pull for evidence Require the publication of the evidence base Require spending bids to provide evidence base Submit government analysis to external expert scrutiny Provide open access to information Facilitating better evidence use Encourage better collaboration across analytical services Co-locate policy makers and internal analysts Integrate analytical staff at all stages Link R&D strategies to departmental business plans Cast external researchers more as partners than as contractors Second more university staff into government Train staff in evidence use Source: Abstracted from PIU 2000, Bullock et al (2001)

34 UK Government “Tools” Overview and Checklist
1. Impact Assessment and Appraisal: guidance checklist for policy makers. Strategy and Policy Evaluation 2. Strategy Survival Guide 3. Magenta Book: Guidance notes on Policy Evaluation 4. Green Book: Appraisal and evaluation in Central Government 5. Regulatory Impact Assessment (RIA) Ensuring Key Perspectives are Included 6. Incorporating regional perspectives into policy making toolkit (Subnational) 7. International Comparisons in Policy Making Toolkit 8. Gender Impact Assessment: a framework for gender mainstreaming 9. Managing risks to the public: Appraisal Guidance Testing Policy Ideas 10. Policy Pilots Public-orientated Tools 11. Concern Assessment Tool 12. Community Engagement How to Guide 13. Connecting with Users and Citizens Getting Better Advice and Evidence 14. Expert Advisory Bodies for Policymakers 15. Improving Standards of Qualitative Research

35 Regulatory Impact Assessment
Aims to improve causality between evidence and advice A process that must be completed for all proposed policy changes: Purpose / intended effect Policy problem Options & evidence Impact & evidence Results of consultation Published More at: office.gov.uk/regulation/ria/ria_guidance/index.asp

36 Using Qualitative Research
A framework developed by the Cabinet Office / National Centre for Social Research Based on review of 29 existing frameworks (esp from medical/health) Four principles. Research should be: contributory; defensible in design; rigourous in conduct; credible in claim. 18 Questions, with criteria Recognises need for: Policymakers to have necessary expertise New approaches to research

37 Using Qualitative Research
How credible are the findings? How has knowledge or understanding been extended by the research? How well does the evaluation address its original aims and purpose? How well is the scope for drawing wider inference explained? How clear is the basis of evaluative appraisal? How defensible is the research design? How well defended are the sample design/target selection of cases/documents? How well is the eventual sample composition and coverage described? How well was the data collection carried out? How well has the approach to and formulation of analysis been conveyed? How well are the contexts of data sources retained and portrayed? How well has diversity of perspective and content been explored? How well has detail, depth and complexity of the data been conveyed? How clear are the links between data, interpretation and conclusions - i.e how well can the route to any conclusions be seen? How clear and coherent is the reporting? How clear are the assumptions/theoretical perspectives/values that have shaped the form and output of the evaluation? What evidence is there of attention to ethical issues? How adequately has the research process been documented? More at:

38 “Tools” for donors… Funding mechanisms Funding communications etc
Competition vs Support Academic rigour vs Policy relevance Individual Organisations vs Partnerships Project vs Programme Outputs vs Outcomes “Home” capacity vs Southern capacity-building Funding communications etc Funding networks Evaluation

39 Evaluation Classical case studies… Episode Studies… IFPRI, IDRC
Can capture depth and diversity Overestimate role of research Episode Studies… ODI Focus on policy change Historical Underestimate role of research

40 Other Approaches Outcome Mapping RAPID Outcome Assessment
Social Network Analysis Most Significant Change “Impact Box” Peer review Expert review

41 Outcome Mapping OUTCOME MAPPING: Building Learning and Reflection into Development Programs Sarah Earl, Fred Carden, and Terry Smutylo

42 RAPID Outcome Assessment
Combining Case Study Episode Study Retrospective Outcome Mapping Involving all stakeholders With ILRI / CGIAR

43 Policy process ranking
What works in DFID? Small workshop with 7 staff. Participatory pair-wise ranking of factors in key policy processes. Worked quite well. Policy processes work if they are: at the right time championed by a senior person based on good theory & evidence solve a problem have budget for implementation

44 Conclusions To improve research impact you need: Clear intent
A systematic approach The right incentives / culture The right systems To spend more on communications To engage, engage, engage To produce the right products for the right people at the right time To look to the long term

45 Further Information ODI – www.odi.org.uk RAPID - www.odi.org.uk/rapid
Publications Working Papers Briefing Papers Books Case Studies Workshops and Seminars Tools and Toolkits Contact:


Download ppt "John Young: ODI, London j.young@odi.org.uk Making Knowledge Count Maximising the value of Research for Development John Young: ODI, London j.young@odi.org.uk."

Similar presentations


Ads by Google