So what? Using Logic Models to Write Strong Impact Statements Katherine Webb-Martinez Program Analyst, MPA Office of Program Planning and Evaluation December 9, 2010

Presentation Desired Outcomes An understanding of how basic logic model techniques help define outcomes An understanding of how to write strong impact statements

What will show that you’ve arrived? Logic Model as Road Map Where are you going? How will you get there? What will show that you’ve arrived? “If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra 3

Everyday Example of Program Theory U N G R Y Get food Eat food Feel better Adapted from University of Wisconsin-Extension, Program Development and Evaluation 4 4 4

Logic Model - chain of connections What we invest The problem What we do What results Situation: Background, Rationale, Goals Inputs: Time, Volunteers, Research base Outputs/Methods: Activities, Participation Products Learning Outcomes: Knowledge, Attitude, Skill Action Outcomes: Behavior, Policy Condition Outcomes: Economic gain, Societal or Environmental improvement Time 5 5

ANR Thematic Logic Model Examples Review in DoView application

Developing a logic model Card Sort Activity: Each set of cards depicts a program Put cards in logical order depicting the program’s theory Adapted from University of Wisconsin-Extension, Program Development and Evaluation

Outputs/Methods How did you serve your clientele? Workshops, short courses, field days, etc. Educational presentations Publications Research findings New methodologies and models New products (patents, plant or animal varieties) New practical knowledge for policy decision makers 8 8

Outcomes How has the clientele/area situation improved as a result of your program? Learning Outcomes ~ short-term Knowledge Attitude Skill Action Outcomes ~ medium-term Behavior/Practice Policy Condition Outcomes ~ long-term Social/Health Economic Environmental 9 9

Activity: Developing your Outcome Chain Adapted from University of Wisconsin-Extension, Program Development and Evaluation

Mark key outcomes Notice patterns of arrows Notice key links Are all activities connected to outcomes? Mark outcomes that are most important to your clientele Mark outcomes that are important to you and the organization Highlight all ‘through-lines’ that go from activity to long-term outcomes – key pathways

What is an Impact Statement? A brief description of the difference your work has made. 12 12

Not how many worms the bird feeds its young, but how well the fledgling flies (United Way of America, 1999) 13 13 13

Writing Outcomes/Impact Statements 1. How have program participants benefitted from your work? 2. What is or will be the ultimate benefit realized (by a group, community, population, area or society at-large)? What effect did your publications have? Was your work adapted and extended by others? What did participants learn? What attitudes changed? What skills were increased? What practices/behavior changed? How many people changed? Were policies changed as a result?  How much money was saved? What were the final outcomes? 14 14

How to Include Long-term Impacts ? Challenges: Take a lot of time to be realized Difficult to measure What you can do: Include desired long-term impacts Use literature/ research to explain how your work connects/contributes to impacts 15 15

Examples of How to Use Literature/Research “Research shows that engaging youth in civic activities is the most effective way to promote civic identity formation and subsequent civic engagement in adulthood.” (UC Cooperative Extension brings new opportunities to Placer County youth by Shannon Dogan) Research shows that planting cover crops reduces erosion. 16

Formula for Strong Outcomes/Impact Statement Describe Outcome/Benefit Explain to Whom or What Area Include Quantification Be sure to answer So What? Who cares? Can also include: Scope of the impact (county, regional, statewide, etc.) Information on how evidence to document outcomes was collected (surveys, observation, etc.) 17 17

Outcomes/Impact Statement Weaknesses Focused on personal not clientele/area outcomes (“proud to be with CE”) Vagueness (“thousands”) Lack of projected impacts to explain program theory (does NOT answer “so what?”) 18

Activity: Review following examples to identify strengths and weaknesses of real impact statements.

Natural Resource Example: More healthy milk for humans.  20 20

Ag Example: I am particularly proud to represent UC ANR in the counties I serve and I believe my program has raised the awareness of Cooperative Extension and the unique role we all play in improving the environment, the economy, and the individual lives of our clientele. I will continue to seek out partnerships and collaborations with UC ANR faculty, specialists, and advisors in order to bring the best science to the issues my clientele face. 21 21

Natural Resource Example: These are new control methods for application on western juniper. They have been adopted by local ranchers and chipping operators. Potentially these techniques could be widely applied on rangelands throughout northeastern California and southern Oregon. 22 22

Natural Resource Example: This research will help managers and policy makers make sound decisions regarding watershed management and policy as it relates to stream temperature. Data should be applicable throughout northern California, the Sierras, as wells the intermountain west. 23 23

NF&CS Example: Based on results from a comparison of pre- and post-Food Behavior Checklist responses from workshop participants who completed four or more hours of lessons in 2004-2005, improvements were documented in the following areas: Food Resource Management: (plan meals, economical shopping and does not run out of food)--72% and Food Safety Practices: (thawing food and food stored properly)--66%. 24 24

Resources University of Wisconsin Extension www.uwex.edu/ces/pdande Logic model templates: http://www.uwex.edu/ces/pdande/evaluation/evallogicmodelworksheets.html Free online course: http://www.uwex.edu/ces/lmcourse/# Penn State Cooperative Extension http://www.extension.psu.edu/evaluation/ American Evaluation Association www.eval.org