Introduction to Monitoring and Evaluation – Measuring Success Cheryl Cichowski Kira Rodriguez Michelle Mitchell

MonitoringEvaluation Measuring success 2

The difference between monitoring and evaluation 3 MonitoringEvaluation When is it done?ContinuouslyOccasionally What is measuredEfficiencyEffectiveness Who is involved?Internal staffExternal contractors What sources of information are used? Internal documentsInternal and external documents Who uses the results?Managers and program staff Mangers, staff, donors, beneficiaries, etc How are the results used?To make minor changesTo make major changes in policy, strategy, and future work

Group Exercise: Monitoring or Evaluation? Attendance at the District Coordinating Council meeting Completion of milestones Policy and / or environmental changes in the community Correct use of funding Representation of Advisory Council Change in knowledge, attitudes, beliefs and skills Number of strategies implemented 4

Monitoring A continuous process of reviewing information gathered from program activities to compare the efficiency of the activities in meeting strategies. Monitoring helps focus the direction of program activities and facilitates minor changes.

Some Areas of State Monitoring Milestones Quarterly Reports Quarterly Counts Resources Success Stories

Why your Project Officer Monitors Compliance with contract Provide TA when needed State program needs to show work progress to federal funders Learn from the grantees and share with others Facilitate a network of prevention providers

Everyday Evaluation Labor Day Weekend activity 8

Evaluation A systematic and objective assessment of an ongoing or completed program, its design, implementation and results. 9

10 Engage Stakeholders Describe the Program Focus the Evaluation Design Gather Credible Evidence Justify Conclusions Ensure use and lessons learned Accuracy UtilityFeasibility Propriety

Overarching Evaluation Questions How well does the HMP Initiative help strengthen Maine’s public health infrastructure? How well is the HMP Initiative being implemented at community and state levels? What impact does the HMP Initiative have on the health status of Maine residents?

Data Collection Coalition Characteristics & Functioning Survey Objective Tracking Environmental Indicators Survey Secondary Surveillance Datasets

Coalition Characteristics Survey 13

Objective Tracking 14 Objective Database Breathe Easy Maine Tobacco- Free College Network Maine Tobacco- Free Hospital Network Department of Labor Maine KeepMEWell HMW

Environmental Indicator Surveys Sample Results from 2011 Muni Survey 15 RESA: How familiar are you with HMP? (All Respondents) Count% $RESaVery Familiar 286% Somewhat Familiar 20842% Somewhat Unfamiliar 11022% Very Unfamiliar 14530% DK/NO ANSWER 51% Total % RESB: Do you know who your local HMP is? (All Respondents) Count% $RESbYes 9720% No 39380% DK/NO ANSWER 71% Total % RESC: Have you had any interaction with your local HMP in the past year? (All Respondents) Count% $REScYes 9119% No 39581% DK/NO ANSWER 112% Total %

Surveillance 16

Ensuring use of findings Reports Data sheets Presentations Website Peer Journals Annual Conferences Display tables

18 5 CLICKS TO EVALUATION DOCUMENTS HMP Grantee ResourcesResources and DocumentsUseful Documents Evaluation Resources

19 Contact Information Please contact us with any questions Cheryl: Kira : Michelle: