1. Helping Others Catch the Vision of Program Evaluation

3. What do you hope to learn today? Hello…

4. Value & enjoy evaluation Create logic models Improve performance measures You CAN help others:

5. Epidemiology & Laboratory Capacity (ELC)

6. ELC Activities Cross-Cutting: Epidemiology Laboratory Health Information Systems Evaluation Disease-Specific: Foodborne Healthcare-Associated Infections Influenza West Nile Virus/Arboviral Vaccine-Preventable

7. Purpose: build evaluation capacity in state public health departments ELC Evaluation Pilot Project

8. Getting started

9. Training Creating logic models Improving performance measures Utah’s Pilot Year

10. Don’t reinvent the wheel

11. Use & share resources

12. www.cdc.gov/eval

13. CDC Evaluation Framework

15. Engage Stakeholders

17. Traveling companions

20. Be “the carer”

21. Newsletter

22. Listen

24. Time it right

25. Learn where are they now? where are they going?

26. What’s the motivation?

27. Learning the language

29. Increasing knowledge

30. Be a resource, not the expert

31. Have fun!

33. Powered by Do Local Health Departments use UDOH disease plans?

34. When LHDs have questions about communicable disease, the disease plans are their #1 source for answers. Answered: 23 Skipped: 0

35. Most LHD respondents use disease plans at least weekly. Answered: 21 Skipped: 2

36. Powered by “my main guide every day” “Our staff uses them constantly” -LHD respondents

37. Connect to purpose

39. Remember the big picture…

40. …but start small

41. How could you help this person “catch the vision”?

42. Describe the Program

44. Program Description

45. Who are you? What do you do? Why does it matter?

46. Example program description

47. Logic Models

48. are maps

50. Credit: Tom Chapel with Stephanie Evergreen

52. ELC Logic Model

53. Logic Model - complete Outcomes or Impacts Inputs Activities Outputs Short-Term Outcomes Intermediate Outcomes Long Term- Outcomes What program needs What program does Who or what will change because of program Context & Assumptions (External factors that influence getting to outcomes)

54. Outcomes Inputs Activities Short-Term Outcomes Intermediate Outcomes Long Term- Outcomes What program needs What program does Who or what will change because of program Logic Model - simplified

55. Activities, Outputs & Outcomes Activities What does the program DO? Provide foodborne outbreak response training for LHDs. Outputs Tangible products of activities Number of trainings held by 7/31/15. Outcomes Who or what will change? Increased LHD knowledge of sampling techniques. Increased number of food samples submitted from LHDs to UPHL.

56. Linking activities to outcomes

57. What do you do? (Activities) Why does it matter? (Outcomes)

59. Outcomes Inputs Activities Short-Term Outcomes Intermediate Outcomes Long Term- Outcomes What program needs What program does Who or what will change because of program Create a logic model

61. Influenza logic model InputsActivities Short-Term Outcomes Intermediate Outcomes Long-Term Outcomes Personnel: – Utah Department of Health staff – Hospital staff – Laboratory staff – CDC staff Laboratory: – Samples – Supplies – Equipment Data and Systems: – Data collection system – Reports from hospitals and laboratories Participate in federal surveillance programs Collect, analyze, and disseminate data Ensure that appropriate laboratory testing is available Communicate with other public health partners and healthcare providers Determine when and where influenza activity is occurring Detect unusual influenza viruses Measure impact of the disease Provide data to guide healthcare decisions Select yearly vaccine components Guide immunization program planning Prevent the spread of influenza Reduce the number of influenza cases Decrease deaths or serious complications due to influenza

62. Focus the Evaluation Plan

64. Who wants to know?

65. What do they want to know? Why?

66. Two question types Outcomes Inputs Activities Short-Term Outcomes Intermediate Outcomes Long Term- Outcomes What program needs What program does PROCESS Questions Who or what will change because of program OUTCOME Measures

67. Gather Credible Evidence

69. Are we there yet?

70. Performance Measures

71. Outcomes Inputs Activities Short-Term Outcomes Intermediate Outcomes Long Term- Outcomes What program needs What program does PROCESS Measures Who or what will change because of program OUTCOME Measures Two types of measures

73. “Good” measures Meaningful & relevant Precise, clear, specific Practical/cost-effective Usable Measurable/observable

75. How would you improve these measures?

76. Justify Conclusions

78. Standards Analysis Interpretation Judgment Recommendations

79. Step 6: Ensure Use & Share Lessons Learned

81. Timing

82. =1,000 words

83. Instant gratification

87.  Epidemiology at any given time is something more than the total of established facts. It includes their orderly arrangement into chains of inference which extend more or less beyond the bounds of direct observation. Such of these chains as are well and truly laid guide investigation to the facts of the future; those that are ill made fetter progress.  A nearly perfect model is John Snow’s analysis of the epidemiology of cholera which led him to the confident conclusion that the specific cause of the disease was a parasitic micro-organism, conforming in all essentials of its natural history to what is now known of the Vibrio cholerae. Wade Hampton Frost, 1936

88. Hypothesis Generation and Testing

89. Share “before & after”

90. Utah Population by Race (US Census Data, 2009 Estimates) Total population Only one race reported White: 93% (80% for US) Black: 1% American Indian/Alaska Native: 1% Asian: 2% Native Hawaiian/Pacific Islander: 1% Two or more races reported: 2% Population under age 5 Only one race reported White: 90% (75% for US) Black: 2% American Indian/Alaska Native: 2% Asian: 2% Native Hawaiian/Pacific Islander: 1% Two or more races reported: 3%

92. Utah is less racially diverse than the US.

93. Generate excitement

96. Value & enjoy evaluation Create logic models Improve performance measures What will YOU do to help others “catch the vision”?

97. Enjoy the journey!