1. Getting on the same page… Creating a common language to use today

2. A couple of questions…  How do you “feel” when you hear the word “evaluation”  What does “evaluation” mean to you?

3. Our definition: Program Evaluation …the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming. Utilization-focused program evaluation is evaluation done for and with specific, intended primary users for specific, intended purposes. (Patton, 1997)

4. What’s that mean?  Systematic collection  Activities, characteristics, and outcomes  Make judgments  Improve effectiveness  Inform decisions  FOR and WITH  Primary users  Specific, intended purposes

5. Two Basic Types of Evaluations  Formative – strengthen or improve the program  Summative – summarize the program for accountability

6. Summative Evaluations  Outcome Evaluation – What outcomes are associated with the program? Are they what we planned?  Impact Evaluation – Did the program cause those outcomes?  Others - Cost-benefit/Cost- effectiveness Analysis; Meta-analysis; Meta Evaluation

7. Formative Evaluations NNeeds Assessment – Is there a need for a program? What type? DDesign Evaluation – Does the design of the program look logical? EEvaluability Assessment – Is the program ready for a summative evaluation? IImplementation Evaluation – Was the program implemented as planned?

8. So Why are We HERE?????  Impact of our programs is difficult to measure  One time (or even few time repeat) visitors or contacts  Short duration of visit  Difficult to create “causal” measures

9. BUT!!!!!!!  If we all share a BIG goal…  We could look at how our programs all work together within the community  And in doing so, gain a sense of impact

10. AND????? JJust how do we do this?

11. We start by creating a “model” of the collective programs  For this project: A collective model for each community (Planning Grant) A sharing between two communities on their models (Planning Grant) Training in modeling (Year 1 if funded) Collective modeling (Year 1 if funded) Measurement of Impact (Year 2 if funded)

12. So What’s This “MODEL?”  A logic model is a tool for examining a program so that we see how it “logically” fits together  We use the logic model to apply “program logic” which says there is a way of measuring a program using causal linkages  We use program logic based on “Program Theory” which explains these causal links

13. A logic model… Specifies cause and effect relationships. CONTEXT Resources Activities Outputs Audiences Outcome 1 Outcome 2 Outcome 3 Outcome 4

14. ` Links the strategy with its intended outcomes. CONTEXT Resources Activities Outputs Audiences Outcome 1 Outcome 2 Outcome 3 Outcome 4 STRATEGYINTENDED OUTCOMES

15. A logic model… Ties together the key theories supporting the logic of the program. CONTEXT Resources Activities Outputs Audiences Outcome 1 Outcome 2 Outcome 3 Outcome 4

16. A generic logic model The Strategy Short and Long Term Outcomes of Employing the Strategy It’s a diagram of the theory of how a program is supposed to work

17. And this leads us to…  If numbers of people change behavior…  And they encourage larger groups to change…  Then there is a greater chance that conditions related to the long-term mission will also change (social, economic, environmental, political).

18. Basic Definitions  Assessment  Evaluation  Input  Output  Outcome  Impact

19. In other words…  Within a certain context  Resources are used to create  A set of activities/tactics  Designed to produce outputs – products or services  For predetermined audiences.  These audiences react to the outputs.  If positive, reactions are more likely to lead to an increase in knowledge, a positive attitude, new skills, and aspirations to change behavior.  If these short-term outcomes occur, behavior is more likely to be sustained or changed.