1 Theory of Change Chesapeake Bay Funders Network Program Evaluation Training Workshop OMG Center for Collaborative Learning January 9-10, 2008

2 Building an Evaluation Plan I. Theory of Change/Pathway Map II. Outcomes III. Indicators IV. Methods V. Putting Evaluation to Work for You

3 To evaluate how well you’re doing, you must have some place you’re trying to get to. Or, in Yogi Berra’s words, “If you don’t know where you’re going, you’ll end up somewhere else.”

4 People hold beliefs or theories that guide our analysis of problems and the way that we respond to them. Clarity about the problem and the theory about how to address it is critical - these beliefs guide our work. Helps us to plan our work as well as evaluate it. Why Theory of Change?

EXAMPLE: Injury Free Coalition for Kids Dog Bite Prevention Program Strategic Focus To reduce dog bites among children in Providence through a pilot dog bite prevention and literacy program Long-term Outcomes The number of emergency room visits by children due to dog bites is significantly reduced. Short-term Outcomes 90% of participants are significantly more accurate at determining safe and unsafe dog situations Program is expanded to include additional classrooms Activities Develop a three session program that educates students about dog bites Deliver program to six elementary schools Assumptions Contextual Analysis In Rhode Island, dog bites are the leading cause of emergency room visits by children In other communities, evidence has shown that educating children about the proper way to interact with dogs reduces the number of bites that occur Assumptions

6 What is a Theory of Change? A theory of change is an individual, group of individuals, or organization’s belief about how to positively change conditions or behaviors It is based on assumptions about what is needed to make these changes Sound theory should be based on experiences and logical judgments about what works The process of articulating the assumptions, actions and strategies for change and illustrating them visually is often called pathway mapping

7 The Elements of Pathway Mapping: The Way it Works 1.Define key problems/issues challenging our communities and constituencies 1.Describe what we hope to change (desired outcomes) in our communities through our interventions 2.Describe how (strategies, actions, interventions) we hope to accomplish those outcomes 3.Articulate our underlying assumptions about the way things work

8 The Elements of Pathway Mapping: The Way it Works Examining assumptions about what works…. While assumptions are often based on experiences from the field, sometimes they are built on intuition. Challenging these assumptions allows organizations to look more closely at their theory and reflect on the choices that they are making. As we discuss our pathways to change, surface and discuss the assumptions that we are making concerning why a particular intervention might work. Challenge those assumptions – what experiences or information are they based on?

9 Linking strategies, actions and desired outcomes…. In order for a pathway map to be useful, a logical path must be created that connects strategies and actions to reasonable and measurable outcomes. Closely examine the links between our strategies, actions and outcomes and ensure that they are logical, achievable and direct. Identify and address gaps in our strategies – paths that might not get us where we want to go. The Elements of Pathway Mapping: The Way it Works

Here’s the model that will help us create our own pathway maps Strategic Focus The overall approach to conducting your work. If you cannot implement all of the desired strategies at once, prioritize them. Long-term Outcomes Positive changes that you hope to achieve in the long- run. Short-term Outcomes Identify the short- term changes in your target entity (water body, population, etc.) that you hope to achieve by your program/activities. Actions/Activities Activities are the individual projects that your organization works on: this could include water testing, organizing clean up events, or contacting legislators. Assumptions Contextual Analysis Identify the major conditions that serve as the backdrop to your work: what are the opportunities and challenges surrounding your work? Assumptions

IFCK EXAMPLE WITH ASSUMPTIONS Strategic Focus To reduce dog bites among children in Providence through a pilot dog bite prevention and literacy program Grantee Activities Develop a three session program that educates students about dog bites Deliver program to six elementary schools Contextual Analysis In Rhode Island, dog bites are the leading cause of emergency room visits by children In other communities, evidence has shown that educating children about the proper way to interact with dogs reduces the number of bites that occur Assumptions Children can be taught behavior-modifying material in three sessions Schools will be receptive to this type of programming Programming learned in the classroom will be applied to situations in real life Short-term Outcomes 90% of participants are significantly more accurate at determining safe and unsafe dog situations Program is expanded to include additional classrooms Long-term Outcomes The number of emergency room visits by children due to dog bites is significantly reduced.

EXAMPLE: Fullojunk Watershed Alliance Strategic Focus Improve the environmental quality of the Fullojunk Watershed through clean up efforts, education, and advocacy. Long-term Outcomes Short-term Outcomes Decreased tons of trash captured in netting system Alliance Activities Organize watershed clean up events Provide education programming to schools and other orgs Contact lawmakers to advocate for policies that benefit the watershed Assumptions Contextual Analysis The watershed is in a highly populated area, so there is a large amount of pollution in the water The local public and policymakers have limited knowledge of the state of the watershed Current legislation does very little to protect the Fullojunk Watershed Assumptions Educating kids and community members about effects of trash on water quality will lead to changes in their behavior

13 How do we develop a Theory of Change 1. Convene all stakeholders for a planning meeting (staff, Board, volunteers, key partners, community members) 2. Designate a facilitator 3. Address each element of the TOC and reach consensus 4. Share with key stakeholders and revise based on feedback

14 How do we Use Theory of Change As a planning and communication tool Involving stakeholders in planning and helping them get clarity and consensus around program or project activities and outcomes Providing opportunities to correct faulty underlying assumptions and modify the program’s design before it is launched

15 How do we Use Theory of Change As a planning and communication tool (cont.) As a check to help inform decisions about changes as the program evolves – Is this within our focus? – Will it help us get to out outcomes? – What are our assumptions?

16 How do we Use Theory of Change As an evaluation tool Laying the foundation for the evaluation plan through the identification of outcomes that will be measured Helping stakeholders in a partnership decide what outcomes they are responsible for

17 How do we Use Theory of Change As an evaluation tool (cont.) Providing a map to revisit implementation steps to inform your understanding of program outcomes (Why did this result happen?)

18 Exercise Use the blank pathway map worksheets in your folder to develop the Contextual Analysis, Strategic Focus, Assumptions, and Activities for your own organization.