1. Evaluation Assists with allocating resources what is working how things can work better

2. Effective Evaluations ID strategies/initiatives working best Provide feedback for decision making Show cost-effectiveness of different strategies Increase ability to manage projects/programs

3. Why Evaluate? To determine effectiveness understand or verify impact of service, program, product To assess efficiency ensuring optimal use of time & resources To be accountable account for what has been accomplished through project funding To identify ways to improve a program Improving usefulness of the program or service Determining what works & doesn’t (strengths & weaknesses) Determining if program is meeting needs of participants

4. Types of Evaluation Formative Information you collect to help you plan or implement a program/service Needs assessment Feasibility assessment Pre-testing program elements Audience analysis

5. Types of Evaluation Process Assesses aspects of program activities On-going process – feedback for decision making along the way Tracking quantity & type of service Tracking people reached (# and type) Participant satisfaction (e.g., quality of service) PIE focus Perfect, Improve, Eliminate

6. Types of Evaluation Impact Whether & to what extent a program has accomplished stated goals & objectives

7. Fiscal or Efficiency Evaluation Compares costs -- economic resources consumed by program WITH consequences -- to health effects or outcomes

8. Fiscal Evaluation Cost-Benefit Analysis Compares cost in $$ economic benefits/consequences in units of currency For every $1 spent on preschool program, there are $7 in benefits through reduced societal costs

9. Fiscal Evaluation – Cost-Effectiveness Analysis Compares: costs in monetary units -- $$ health effects in units of mortality, morbidity, health status measure Costs $x to prevent one infant death from diarrheal disease by supporting breastfeeding

10. Fiscal Evaluation Cost-Utility Analysis Compares cost in $$ consequences in health terms as utilities or the preference/desire for a specific outcome Compare costs and outcomes of interventions Cost $x to avoid one case of diarrheal disease through oral rehydration or clean water supply

11. Steps in Evaluation DESCRIBE/CLARIFY YOUR PROGRAM Engage stakeholders Assess resources DESIGN THE EVALUATION Determine evaluation methods Collect the data Process and analyze the data Interpret and disseminate the results TAKE ACTION Who conducts evaluation?

12. Evaluation Framework Provides a plan that links the activities (implementation objectives) and outcomes (outcome objectives) with your evaluation questions, indicators, data sources and data collection methods Evaluation questions directly reflect program’s implementation and/or outcome objectives.

13. Evaluation Framework Indicators What are the specific measures indicating you have achieved your objectives? How will you know if you accomplished your objectives? What would be a success? Sources Who can provide information about this? Where can you get the information? Tools What is/are the best way(s) to gather information/find out from people?

14. Implementation or outcome objective : Evaluation Questions IndicatorsData Sources Data Methods Evaluation Framework

15. Measurement Indicators Outcome Indicators measures for your outcomes Short-term Improved access Improved collaboration Policy changes Changes in awareness, knowledge or beliefs Longer term Service utilization Morbidity/mortality Health status

16. Measurement Indicators Process Indicators Measures for how program delivered Training sessions held Type of staff Staff activities and time Resources distributed Participation Client Satisfaction

17. Program Logic Model A diagrammatic representation of a program Depicts relationships between program activities, outputs & outcomes Describes a program to stakeholders clarifies how the program is structured Shows how different facets of a program are linked Integrates program planning & evaluation Facilitates the assessment of the feasibility of program activities achieving program objectives

18. Program Logic Model – CAT SOLO Components – Closely related groups of activities in your program Activities – describe what you are going to do & provide (implementation strategies) Target – describes who you are trying to reach Outcomes – describe impacts on the customers, clients, organizations, systems (short and long-term outcomes)

19. Program Logic Model – CAT SOLO Components Activities Target Short-term Outcomes Long-term Outcomes

20. Communicating Evaluation Findings Communicate info to potential users Ensure report addresses issues users perceive as important Ensure timely delivery Ensure format easily understood by users

21. Communicating Evaluation Findings Include copies of survey tools Summarize computer printouts in table format

22. Potential Users of Evaluation Results Immediate Supervisor Senior Management/Boards of Directors Policymakers Funding agencies Peers in similar programs Client base/Community Program personnel

23. Communicating Evaluation Findings: Figure 4-4, p. 123