1. M&E Plan Overview M&E Capacity Strengthening Workshop Addis Ababa
4 to 8 June 2012
Arif Rashid, TOPS

2. What is an M&E Plan? An M&E plan is:
A flexible guide to the steps needed to:
Inform and guide monitoring and evaluation activities;
Document, report, and communicate project activities;
and illustrate progress toward achievement of project goals and objectives.
Slide # 1

3. An M&E Guide Tailored to your Project
What is an M&E Plan?
An M&E Guide Tailored to your Project
Slide # 2

4. What are the components of the M&E Plan?
M&E goals and objectives
M&E methodologies & implementation plan
Data collection procedures
Data management and analysis procedures
Reporting and communications plan
Description of M&E activities
Output monitoring
Process monitoring
Outcome monitoring
Monitoring of environmental indicators
Commodity end-use monitoring
Slide # 3

5. What are the components of the M&E Plan?
Calendar of M&E activities
Data quality assurance plan
Data collection instruments
Organizational structure of M&E unit
Communication strategy and use of data in decision making
Budget for M&E activities
Slide # 4

6. M&E Plan components Focus of this workshop M&E Activities
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 5

7. Unpacking the M&E Plan: Key elements
Slide # 6

8. M&E Plan components A M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy A
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 7

9. A. Results framework (RF)
“A results framework …[is a]…type of logic model which represents the food aid program’s theory of change by laying out the activities and outputs that will lead to short, medium and long term outcomes and objectives”
Source:
Slide # 8

10. RF example GOAL SO 1 IR1.1 Activities IR1.2 SO2 IR2.1 IR2.2 SO3 IR3.1
Slide # 9

11. M&E Plan components B M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan B
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 10

12. B. IPTT
“The indicator performance tracking table (IPTT) …includes performance indicators (at the impact, outcome and output levels) linked to the food aid program proposal’s objectives, and baseline and target values for each indicator (estimated)”
Source:
Slide # 11

13. FY2012 IPTT Template
Slide # 12

14. M&E Plan components C M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools C
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 13

15. C. M&E Matrix / PIRS/PMP
A performance management plan (PMP) includes: “definitions for each indicator listed in the IPTT, the data collection methodology and frequency”
Source:
Slide # 14

16. C. M&E Matrix / PIRS/PMP
An M&E matrix (also known as an Indicator Matrix/ or Performance Indicator Reference Sheet):
summarizes of detailed information for each project indicator.
serves as a helpful tool for planning and managing data collection, analysis and use.
expands on the IPTT/PMP to identify key information required for each indicator and summarize the key M&E tasks for the project.
Adapted from: Chaplowe, Scott G. (2008)
Slide # 15

17. M&E Plan components C M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools C
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 16

18. D. Key M&E activities Monitoring Output monitoring
Process monitoring (quality monitoring)
Outcome monitoring
Environmental indicator monitoring
Commodity end-use monitoring
Evaluation
Reporting and communications
Review of M&E process
Slide # 17

19. Output monitoring
Reporting and tracking project outputs to compare with targets and resource used
Information Source
Project reporting
M&E plan needs to develop:
Reporting forms
Data base(s)
Slide # 18

20. Process monitoring
Monitoring the “quality” of services delivered to beneficiaries
Information Source
Surveys of “final service providers”
M&E plan needs to develop:
Sample design
Instruments
Analysis plan
Slide # 19

21. Outcome monitoring
Monitoring outcome-level changes of project beneficiaries and institutions
To what extent have project interventions altered
Knowledge
Perceptions, and
Behaviors
of beneficiaries
Slide # 20

22. Outcome monitoring To what extent have project interventions altered
Institutional competency
Policy change or
Quality of service changes.
Information source
Surveys of beneficiaries
M&E plan needs to develop for outcome monitoring:
Sample design
Questionnaire
Analysis plan
institutional competency (e.g., improved health-care systems),
policy change (e.g., new or revised policies, change of enforcement) or
quality of service changes (e.g., more effective extension system).
Slide # 21

23. Environmental indicator monitoring
Monitoring the indicators identified in the IEE as negative determination with conditions
Information Source
Based on the proposed mitigating measures in IEE
M&E plan needs to develop:
List of indicators and mitigating measures
Instruments
Analysis plan
Slide # 22

24. Commodity end-use monitoring
Monitoring end-use of the commodities distributed by the project
Information Source
Sample survey of beneficiaries that received commodities from the project
M&E plan needs to develop:
Sample design
Instruments
Analysis plan
Slide # 23

25. Evaluation activities
Evaluating project impacts should be taken into consideration from the inception of the project. Project evaluation is composed of three separate events that should be considered parts of a whole. [Note: Food for Peace does not require a quantitative survey during the mid term review]
Baseline
Mid-term
Final
Slide # 24

26. USAID Evaluation policy
“In most cases, the evaluations will be external (i.e., a third-party contractor or grantee, managed directly by USAID, will evaluate the projects), and the contract or grant for the evaluation will be managed by an operating unit’s Program Office.”
Please note that: The Title II Project evaluations are currently managed by the PVOs not by FFP/USAID which will continue until FFP decides otherwise.
Slide # 25

27. Evaluation activities
M&E units are responsible for organizing evaluation activities:
Steps:
Identify study objectives/research questions, target population(s), coverage and reach of study, timeline and budget
Identify specific indicators to be measured
Compile sample frame (from appropriate population)
Using information above, develop SOW for external consultants
Organize logistics for evaluation studies to support external consultants
Slide # 26

28. Review of M&E plan
M&E plan should be regularly reviewed to assess whether the current plan is meeting objectives and needs of project:
Reporting needs to donor
Project management needs
Review should assess:
Indicators
Data collection and analysis procedures
Communication strategy
Slide # 27

29. M&E Plan components E M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools E
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 28

30. E. M&E Tools and methods M&E tools and methods include:
Project monitoring reporting forms
Database formats and management procedures
Survey sampling strategy and analysis plans
Reporting processes and procedures
Slide # 29

31. M&E Plan components F M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline F
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 30

32. F. M&E timeline
An M&E timeline is “a schedule of M&E events, describing procedures and timing for analysis and use of performance data in order to modify activities based on lessons learned”
Source:
Slide # 31

33. M&E Timeline example Slide # 32 General activities M&E Staff hiring
Year 1
Year 2
Year 3
Year 4
Year 5 Q1 Q2 Q3 Q4
General activities
M&E Staff hiring
Equipment procurement
Baseline survey
Baseline results reported and disseminated
Mid-term evaluation
MTE results reported and disseminated
Final evaluation
Final evaluation results reported and disseminated
SO 1 activities
Project monitoring/reporting
SO 2 activities
SO 3 activities
Slide # 32

34. M&E Plan components G M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy G
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 33

35. G. Communication Strategy
An important (and often overlooked) component of M&E plans is how to disseminate and use the information obtained by the M&E system
Although the M&E Officer or Unit does not have much control over the managers or the technicians to ensure that M&E data inform the programmatic decisions, M&E unit can facilitate the process. M&E
Slide # 34

36. G. Communication strategy
Careful thought needs to be given to disseminating and using M&E information including:
What information should be distributed?
Who needs what information?
Why do they want this information?
How much information they want to know?
How does the information get distributed?
Source: Family Health International (2004)
Slide # 35

37. M&E Plan components H M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing H
M&E Budget
Data Quality
General implementation plan
DIP
Slide # 36

38. H. M&E Staffing plan
Based on information needs outlined in earlier steps, the following capacities may be required for an M&E unit (Pls refer to core competencies for detailed skills):
M&E plan design
Data collection methods (quantitative & qualitative)
Data management skills
Data analysis skills
Data reporting
Project monitoring
Slide # 37

39. Sample M&E organizational chart
(Extensive knowledge of: M&E plan design, data collection methods, analysis and reporting)
M&E Manager
(Project monitoring & reporting, data collection, supervising enumerators, etc.)
M&E Officer
M&E Assistant/ Data entry Clerk
Slide # 38

40. M&E Plan components I M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget I
Data Quality
General implementation plan
DIP
Slide # 39

41. I. M&E Budget
One challenge in developing M&E plans for Development Food Aid Projects is that the M&E budget is normally established before the detailed M&E plan is finalized
As a result, the M&E budget will need to be revisited after the M&E plan is developed
Budget must be aligned with activities in M&E plan
Slide # 40

42. M&E Plan components J M&E Activities PIRS /M&E Matrix IPTT PMP
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS /M&E Matrix
Communication strategy
IPTT
PMP
Results Framework
M&E Staffing
M&E Budget
Data Quality J
General implementation plan
DIP
Slide # 41

43. J. Data Quality: Assurance and Assessment
Data Quality Assurance - A formalized process for defining the appropriate dimensions and criteria of data quality, and procedures to ensure that data quality criteria are met over time
Data Quality Assessment –Review of project M&E system to ensure that quality of data captured by the M&E system is acceptable.
Slide # 42

44. What’s a Data Quality Assurance Plan?
A data quality assurance plan includes activities to improve data quality:
Clear definition of indicators including tabulation plan and measurement protocols
Appropriate design of data collection, aggregation and analysis procedures
Designing data base with appropriate checks and bounds to reduce data entry error
Well designed data management procedures
Follow-up verification of indicator values
Allocate adequate Resources and Skilled Staff
Periodically Assess Data Quality using established protocols
Slide # 43

45. What’s a Data Quality Assessment?
A data quality assessment is a periodic review that:
Helps the project to determine and document “How good are the data?”
Provides an opportunity for capacity-building of implementing partners.
Slide # 44

46. We have now identified all the components of an M&E plan
M&E Methods/tools
M&E Timeline
M&E Activities
- Monitoring
- Surveys
- Secondary data
- Analysis
- Reporting
- Dissemination
- Review M&E Plan
PIRS/ M&E Matrix
Results Framework
Communication strategy
IPTT
PMP
M&E Staffing
M&E Budget
Data Quality
All these components should be in the M&E plan document
Slide # 45

47. Outline for M&E plan Objectives of M&E system Description of
Results framework
IPTT
PMP/M&E Matrix/PIRS
Project staffing and organizational structure
Slide # 46

48. Outline for M&E plan Description of Monitoring Activities Evaluation
Description of processes, tools and frequency
Evaluation
Description of processes and methodology
Reporting and communications strategy
Data Quality Assurance Plan
M&E review process
M&E staffing and organization
M&E budget
Use of information in Decision Making
Slide # 47

49. Outline for M&E plan Annexes Results framework IPTT
M&E matrix/PIRS/PMP
Reporting formats for output monitoring
Questionnaire forms for process/outcome monitoring
Timeline for M&E activities
M&E unit organizational chart
M&E budget
Slide # 48

50. The remainder of the workshop will provide guidance on how to develop these components of the M&E plan
Slide # 49

51. This presentation was made possible by the generous support of the American people through the Office of Food for Peace in United States Agency for International Development (USAID). The contents are the responsibility of Save the Children and do not necessarily reflect the views of USAID or the United States Government.