Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 9, 2012

 Session objectives : Explain similarities and differences between monitoring and evaluation Describe the major purposes, types and tools for MCH program evaluation Contrast traditional evaluation approaches with participatory evaluation Develop a monitoring plan for one program activity

 Improve program planning process  Improve program management  Improve program performance  Assess program effects on beneficiaries  Strengthen links with participant groups

 Periodic, regular collection and analysis of selected indicators  Determines whether key activities are being carried out as planned, and corrections needed  The HMIS (health management information system) provides early indication of progress, or a lack of progress

 Carried out by any staff, not a specific evaluation staff person  Shows strengths & weaknesses of the strategies: do we need to refine them?  Important: use it to provide regular feedback to staff

 We monitor implementation of an intervention – typically output and process indicators  We supervise an individual

Annual totals: 2009 = = 7203

Monitoring and Evaluation: how are they related?

 Monitoring: the regular collection and analysis of selected indicators conducted to determine whether key activities are being carried out as planned, and identify corrections needed  Evaluation: done at longer intervals than monitoring conducted to provide an indication of success of a program, and identify problem areas

 Assesses value Carried out selectively – costly and time consuming Includes a review of evidence  Assess performance and effects of program efforts:  Process: outputs, organization, management  Outcome or impact: effects on beneficiaries

 Strategy – Are the right things being done?  Operations – Are things being done right?  Learning – Is it having the desired effects? Are there better ways?

 Formative vs. summative  Of relative need for a program  Of the feasibility of a program design  Of program performance or process  Of direct effects or outcomes of a program  Of long-term impact  Internal or external

 Staff know more history, organization, culture, problems, successes  Known to the program so less likely to be threatening, disruptive  Can interpret evaluation findings more accurately but:  May be too close, hard to be ‘objective’  Part of power structure, may have personal agendas  Likely not highly trained in evaluation

 Can take a ‘fresh’ look at the program  Not personally involved; more ‘objective’  Not a part of normal power structure so less chance of personal gain  Well trained in evaluation methods, seen as an ‘expert’ by program but:  Outsiders may not understand the program or people involved  May cause anxiety if not known and trusted

 Participation by a range of stakeholders  Focus on program participants, not donors  Focus is on learning, not just accountability  Flexible design, not predetermined  Use rapid appraisal methods, less formal  Consultants are facilitators, not the actual ‘evaluator’  Evaluation team is key to approach

InputsActivities Processes Outputs Objectives OutcomesImpact

Preparation:  Establish purpose, methods (‘scope of work’) including the evaluation questions  Develop data gathering instruments  Develop team plan and itinerary Main activities:  Visit main offices, field sites  Gather data (document review; observations, group discussions, interviews; review surveys)

The evaluation report :  Team discussion of findings, come to tentative conclusions/recommendations  Draft main findings of the evaluation report  Discuss draft of findings with program staff, other stakeholders  Revise findings, recommendations as needed  Finalize and submit report  Share with staff, community, MOH, others  Plan how to implement recommendations

From Halcolm’s Evaluation Laws

Choose one objective for your group project For that objective, select one important activity that the project will monitor Discuss and fill in the information for the six exercise questions