1. Introduction to Participatory Monitoring & Evaluation (PM&E): A Practical Approach to Engaging Stakeholders and Communities in Monitoring & Evaluation Arlene Vincent-Mark, Ph.D., MBA, MA (770) 315.2406 AEA Conference/Atlanta, GA June 12, 2011

2. CDC Evaluation Framework Source: Centers for Disease Control and Prevention (CDC). 1999. “Framework for Program Evaluation in Public Health.”

3. Half the world is composed of people who have something to say and can’t, and the other half consists of people who have nothing to say and keep on saying it Robert Frost

4. Monitoring & Evaluation Monitoring: – routine, continuous assessment of ongoing activities and progress – assesses compliance with work plans & budgets Evaluation: – episodic assessment of overall achievements of program objectives – focuses on measuring whether planned outcomes and impacts have been realized

5. What is PM&E Participatory Monitoring: process where major stakeholders play a key role in tracking progress, and reviewing if project implementation is progressing according to plan Participatory Evaluation: The involvement & participation of community members and other stakeholders in the design and execution of the evaluation process – Reflective – People-centered – Considers stakeholders’ concerns, claims, issues

6. Reasons to Evaluate To make programs more effective To analyze costs in relation to benefits To analyze gaps in performance To share experiences To analyze strengths & weaknesses To compare with other programs To improve plans for the future To see where we’re going and whether there is a need for change To improve monitoring methods To critique our work

7. Evaluation Criteria Effectiveness Efficiency Impact Relevance Sustainability Others?

8. Rationale for PM&E Build stakeholders’ capacity Facilitate joint learning Provide accountability at all levels Celebrate what is working Build sustainability

9. Key Principles in PM&E Stakeholders (local people/community members/project staff) are active participants – not just sources of information Outsiders serve as facilitators Focus on stakeholder capacity building Draw on local resources and capacities Triangulate – (tools/methods, places, times, disciplines, people)

10. Usefulness of PM&E Encourages: – ownership & responsibility – commitment to implementing recommendations Empowers communities Allows for community needs to be considered Provides stakeholders with skills/tools to transform their environment Seen as more flexible and less rigid

11. Advantages of PM&E Practical Useful Formative – Seeks to improve program outcomes Empowering

12. Advantages vs. Disadvantages AdvantagesDisadvantages Less expensive than an external consultant Process may require more time Participants have a say in decision- making Requires more coordination Increased commitment by participants in success of program May be more challenging to facilitate Collaborative process builds & strengthens stakeholders’ relationship Requires investment in evaluation training Results are more likely to be acted on Requires commitment from stakeholders Increases stakeholders’ knowledge and skills in evaluation

13. PARTICIPATORYCONVENTIONAL WHO Stakeholders, communities, project staff, outside facilitator External experts, project staff WHAT Indicators identified by stakeholders to measure process as well as outputs or outcomes Predetermined indicators to measure inputs and outputs HOW Qualitative & quantitative methods by stakeholders Questionnaire surveys by “neutral” evaluators WHY To empower stakeholders to take corrective action To make project and staff accountable to funding agency WHEN Throughout the project periodAt end of project period

14. 14 Strength of PA Compared to Survey Method/CriteriaPASurvey Coverage/Scale of applicabilityMediumHigh RepresentativenessMediumHigh Ease of data standardizationMediumHigh Statistical ComparisonsM to LHigh Ability to capture qualitative informationHighLow Ability to capture causal processesHighLow Ability to capture diversity of perceptionsHighLow Ability to identify felt needs/prioritiesHighLow Degree of participation of the marginalizedHighLow Cost rangeH to M TimescaleH to M

15. Steps in PM&E

16. Ways to Involve Stakeholders Designing the plan Identifying indicators Collecting information/data Analyzing information/data Reporting information Incorporating information & feedback into project

17. When to use PM&E Commitment to participatory process exists Time is not of high importance Funders are supportive Goal is to empower and develop transferable skills

18. Designing a PM&E Plan: Considerations Objectives of project Who participates – Which stakeholders to involve – At what stages to involve them Whose issues should be prioritized How PM&E fits in with overall project What methods/tools to use How to make project responsive to feedback

19. Issue/Sources of Power in PM&E Position Resource access and control Knowledge & skill Social status

20. Be mindful of power…. Respect different cultural, language and communication backgrounds Give voice to the marginalized & vulnerable Be aware of inequality – gender, caste, ethnic identity, religion, etc.) Be sensitive to power relations – Awareness of your behaviors, attitudes and assumptions

21. Common Mistakes in PM&E Insufficient planning & preparation Too much information too quickly Lack of verification of information Losing focus with tools & techniques Insufficient self-reflection All or nothing approach

22. Suggestions to Conducting PM&E Integrate into existing process Be patient Avoid being overly ambitious Consider the different perspectives/opinions Assess costs with benefits Understand unintended and intended impacts Be ready for conflict/differing opinions Enjoy the process!!!

23. 23 Participatory Approaches are about: Actively engaging stakeholders Giving voice to the marginalized, the poor, the disadvantaged Increasing communities’ awareness of issues to decrease health disparities Empowering communities, building capacity, sharing with and learning from partners/stakeholders Final Thoughts

24. 24 Realizing the vision of healthy people in healthy communities is possible only if the community, in its full cultural, social, and economic diversity, is an authentic partner in changing the conditions for health Institute of Medicine (IOM)