1. ADVISORY BOARD MEETING 2013 Doing a Forest Governance Assessment: Practical Tips and Tricks Ken Rosenbaum, Sylvan Environmental Consultants

2. Today we will look at three topics 1.The simplest way to carry out an assessment: take a data collection tool that someone else has designed and adapt it to your needs. 2.How to design your own data collection tool (or modify an existing tool) if you can’t find one that suits your needs. 3.What are good practices in collecting forest governance data?

3. But first a brief recap from Webinar 1: What is a forest governance assessment? An assessment is an attempt to measure forest governance. Assessments can be used to diagnose problems, compare conditions, or monitor efforts to change.

4. FOUR WELL-DOCUMENTED TOOLS AIMED AT FORESTS OR NATURAL RESOURCES The PROFOR Diagnostic. The USAID Sustainable Conservation Approaches in Priority Ecosystems (SCAPES) governance assessment tool. The World Resources Institute’s (WRI’s) Governance of Forests Initiative Indicators The Indonesia Participatory Governance Assessment (PGA) for REDD+.

5. The PROFOR Diagnostic Tool

6. Useful to diagnose problems, to identify priority areas for reform, or to set a baseline for future monitoring Based on the FAO–PROFOR Framework Uses a set of up to 130 indicators (English, French, and Russian versions available) Scored by stakeholders in workshop(s) Validated by key informants Complete “How to” guide available on PROFOR.info website

7. The USAID SCAPES Tool

8. Developed by Wildlife Conservation Society for USAID Designed to work on landscape level, looking at natural resource management generally Based on its own framework for analyzing governance, around pillars of Legitimacy, Capacity, and Power A workshop, bringing together representatives of stakeholders or other key informants or experts, identifies key groups and their influence, strengths, and weaknesses Full how-to guide available on frameweb.org

9. The Indonesia PGA

10. Developed in Indonesia with support of UNDP/UN- REDD as part of REDD+ readiness Not presented as a tool for general use, but as a report on what Indonesia did Highly participatory, steered by an advisory group of experts in close contact with stakeholders Used 117 indicators Gathered data locally and regionally, via combination of methods. including document reviews, key informant interviews, and focus groups. Report available on undp.org site

11. WRI Governance of Forests Initiative

12. An approach piloted in Brazil, Indonesia, and Cameroon Uses 122 indicators WRI has published separate publications on (1) the indicators and (2) methods to score them Can score via desk reviews, key informant interviews, focus groups, participant observation, testing of systems, etc.

13. I’d Rather Design It Myself! 1.Refine your Scope: Exactly What to Measure 2.Identify Sources of Data 3.Select Data Collection Methods 4.Develop “Tools”: Interview Protocols, Questionnaires, Sampling Plans, etc. 5.Write out a Data Collection Plan or Manual

14. Refine your scope: What to Measure Didn’t we set the scope in our Work Plan? Yes, and we’ll start with that, but we need more detail now Look at what other assessments have done Decide how detailed you need to be Decide whether to use a narrative description of scope or to use criteria and indicators Set out your scope in writing

15. Identify Sources of Data Written materials: past assessments, official publications, unofficial government documents, laws, budgets, media reports, academic studies, etc. People: officials, academics, experts, stakeholders, etc. that can be reached through interviews, focus groups, workshops, surveys, etc. Physical evidence: Less commonly used in governance assessments, these can include forest site evaluations, testing of government functions, inspection of boundary markers.

16. Identify Methods Now you have a sense of where the information might be; how will you collect & validate it? Desk reviews: a good way to tap existing documents; low cost; but no “new” information Expert analysis: Can use multiple experts to add depth to desk reviews, provide opinion, validate other methods; but potential for biased experts. Surveys: Produce lots of data; repeatable; but costly; may not be in-depth; may have bias Key informants: may tap rich sources of information at low cost; but may be biased, not easily replicated

17. More methods Focus groups: More people=broader perspective than one-on-one interviews or surveys, but some less assertive people may not be heard. Workshops: Even broader perspectives, promotes communication among stakeholders, but can be expensive; hard to get balance of stakeholders; requires planning and skill to extract information from participants.

18. Develop tools Desk reviews: will you gather existing data or perform new analysis? Experts: What kinds of experts? What terms of reference? Key Informants: How to select? Protocol for interviews? Surveys: Sampling issues. Question design. Focus Groups & Workshops: Sampling issues. How to convene? How to capture (code) responses Get help from social scientists!

19. Write it down A data collection plan: for use of the managers of the data collection effort Maybe just a refinement of the work plan If it is more, check against the work plan, including the timeline and budget, and adjust as needed A data collection manual: for use of the people in the field who are actually collecting data Not always necessary, but in a large effort, helps with consistency

20. Vet & validate your methods Consult experts and stakeholders about your design effort. Do it formally or informally. Be transparent! Transparency now leads to greater credibility later.

21. Good Practices in Data Collection Assembling a data collection team Going out an getting your data Assuring data quality

22. Assembling a team This step varies, depending on methods and size of effort May need just a few people, who take on multiple roles May need managers, logistics coordinators, researchers/field people, data managers, etc. Larger teams need more documentation, training Team members should be trustworthy, capable, and unbiased

23. Collecting data Interviewing, facilitation, and survey administration are all teachable skills. Be sure your team is using good and consistent techniques. Give careful thought to coding: your team should capture data in a complete and consistent manner. Be aware of ethical issues: be transparent and truthful; get informed consent; respect privacy and confidentiality; keep people safe from harm; guard the integrity of the data.

24. Quality assurance When data come in from the field: Edit: Be sure data entries or notes are complete and readable; get back to collectors if not. Clean: Flag data that stands out or suggests an error in entry; look for missing or duplicated entries; etc. Verify: Spot check that data were actually collected; also that data were not miscopied or garbled in transmission. Triangulate: Confirm against other sources.

25. WHERE WE HAVE BEEN & WILL HEAD NEXT: Webinar 1 (January 20) covered the definition of forest governance and how to plan an assessment. A recording is available online. Webinar 3 (26 March) will cover analyzing your data, making recommendations, and getting your results out to the right audiences. Also, we will talk about setting the stage for any assessments that might follow.

26. For More Guidance: The FAO–PROFOR Framework: Search on the internet for “Framework for Assessing and Monitoring Forest Governance” The PROFOR–FAO Guide to Good Practices: Search on the Internet for “Forest Governance Data Collection and Analysis”

27. THE REFORM CHALLENGE “If you want to make enemies, try to change something.” — Woodrow Wilson, President of the Untied States, speaking in 1916.

28. THANKS FOR LISTENING For more information contact: Nalin Kishor, Ph. D Sr. Natural Resources Economist PROFOR Forests Team, GENDR nkishor@worldbank.org The World Bank 1818 H St., N.W. Washington, D.C. 20433  +1-202-473-8672 www.profor.info www.profor.info