1. Step 3: Focus Evaluation Design Luann C. D’Ambrosio, MEd

2. Understand how to focus the scope of your evaluation Describe the difference between process and outcome aims Develop questions to ask for gathering information Understand how to develop standards and indicators to measure progress Develop an evaluation plan for your own program(s) Objectives

3. CDC Evaluation Framework

4. When Designing Your Evaluation, Consider

5. Plan Focus Process (means, resources/ activities/outputs on your logic model) The number of inspections conducted by food inspection staff. Process (means, resources/ activities/outputs on your logic model) The number of inspections conducted by food inspection staff. Outcome (ends, outcomes/goal on your logic model) Reduction in critical item food safety violations. Outcome (ends, outcomes/goal on your logic model) Reduction in critical item food safety violations.

6. How Do I Know What Questions to Ask? Preventing disease Reducing health risk Promoting positive health behaviors Compiling with legal mandates Responding to public complaints Addressing current impact/ threats Anticipating future impact/threats

7. How Do I Know What Questions to Ask? Maximizing use of resources Demonstrating effective public relations Ensuring program sustainability Maintaining program control Improving quality Managing costs Effectively communicating with the public

8. Outcome or Process? Hint: Ask the “Why?” question Preventing Disease Effectively communicating with the public Promoting positive health behaviors OutcomeProcess

9. Using Questions to Develop Standards Use your questions to describe what should exist in order for the program to operating at an optimum level – the “bar.”

10. Process Question Example Standard: Each EH specialist should be inspecting 100 food service establishments per month. Process How many inspections should each EH specialist be performing each month?

11. Outcome Question Example Standard: The average number of critical item violations for food service establishments should be no greater than 2.5. Outcome How can we reduce the risk of foodborne illness in food service establishments?

12. Finding Standards Your stakeholders National, state, local standards Program standards Other jurisdictions Existing

13. Identify a process question, then turn it into a process standard Identify an outcome question, then turn it into an outcome standard Group Practice

14. How Do I Measure Differences? Indicators Ask “What information do I need to determine if my standard is being met?

15. Example Standard: Each EH Specialist should be inspecting 100 food service establishments per month. Indicators: # inspections performed per month # FTE’s in food program conducting inspections Average time to conduct an inspection # of food service establishments in jurisdiction

16. Group Practice Home grown tomatoes to meet your family’s needs 1. Identify family’s needs (standard) 2. Develop an Outcome and Process standard 3. Outcome and Process Indicators Muffet

17. Group Practice Are we able to grow tomato plants for our salad, canning, and juice needs this year? Question Raise 8 tomato plants to maturity by Sept 15. Process standard # plants reaching maturity by Sept 15 # plants producing green tomatoes by July 15 # times fertilized per month # times watered per week # times plants pruned # plants reaching maturity by Sept 15 # plants producing green tomatoes by July 15 # times fertilized per month # times watered per week # times plants pruned Indicators

18. Group Practice Are we able to produce sufficient, eatable tomatoes for our salad, canning, and juice needs this year? Question 50 lbs of ripe, edible tomatoes harvested by Sept 15 Outcome standard # of ripe, eatable tomatoes per plant Pounds of eatable tomatoes picked by Aug 30 Pounds of eatable tomatoes picked by Sept 15 Pounds of tomatoes discarded by Sept 15 # of jars of tomatoes canned by Sept 15 # jars of homemade tomato juice available by Sept 15 # of salads containing tomatoes consumed by Sept 15 # of ripe, eatable tomatoes per plant Pounds of eatable tomatoes picked by Aug 30 Pounds of eatable tomatoes picked by Sept 15 Pounds of tomatoes discarded by Sept 15 # of jars of tomatoes canned by Sept 15 # jars of homemade tomato juice available by Sept 15 # of salads containing tomatoes consumed by Sept 15 Indicators

19. Important Terms to Remember Process Outcome Standards Indicators

20. Questions?