PMEPME This presentation looks specifically at the monitoring of projects/ programmes. It follows on from a more general introduction to PME systems. Version 1 – Avril 2003

Monitoring 1. Importance and risks 2. Approach 1.Stakeholders 2.Indicators 3.Field of observation 4.Criteria 3. Tools 1.For collecting data 2.For analysing information 3.For summarising information 4. Bibliography

Importance and risks

Monitoring is the energising factor in a PME system A project starts with an analysis of the requests made, the needs identified and local resources and know-how. Its programming consists of specifying who ? does what? when ? how ? why ? Monitoring is the periodic supervision of how an action is being implemented The evaluation is for assessing the quality of an action, for drawing conclusions from the experience and for reporting on the action to the stakeholders concerned. The logical framework is the basic tool for presenting a project and working on its programming, monitoring and evaluation

« Monitoring is the art of procuring the information necessary for taking those decisions that govern the orientation of work underway, as quickly and as inexpensively as possible» (GTZ, 1997) Or, to quote Catherine Care on the subject of sharing knowledge : “Stop rowing and start steering” « Arrêtez de ramer et prenez le gouvernail »

The risks (1)  Too much information kills the information! An surplus of information prevents it from being stored accurately. There is not enough time and resources for analysing and utilising the information. Too detailed information can prevent the identification of or hide the important trends DDC, 1997

The risks (2)  Too little information prevents us : from anticipating and therefore mobilising the resources needed for the action; from reporting on the implementation of activities to the parties concerned (beneficiaries, partners, funding bodies, donors…).

Monitoring system

The essential questions to ask The two biggest questions are : who needs the monitoring information? What information is really needed? On this basis, we can determine :  Who should participate in devising the monitoring system?  How should the information be collected, analysed and presented?  What degree of detail is required?  How long and how often should the information be collected and analysed? GTZ,1997

The monitoring system stakeholders During programming and implementation the stakeholders define and measure... indicators for keeping in touch with reality … the indicators for keeping in touch with reality… fields of observation …in the different fields of observation… criteria …according to different angles of analysis (criteria). LOGICAL FRAMEWORK

A - Who are the stakeholders concerned? Those potentially concerned by the definition and implementation of the monitoring system are: Within HI : In the field:project manager, project team, field programme director, administrator… At head office:desk officer, technical co-ordination unit, financial controller, HR service.. Outside of HI : Local stakeholders, Project partners, Local collectivities, Public institutions, technical services Institutional funding bodies …

A relatively complex internal stakeholders ’ map Project manager Directeur de programme Administrator Responsable de programme Financial controller Methods and Techniques Section Directeur opérationnel des programmes RH Logistique Board/ AC Technical management Sections This diagram gives an idea. It is not exhaustive …due to a lack of space Stakeholder Line management Diagram of the flow of information Terrain Head Office And all these people need precise and specific information … And that ’s without counting the external stakeholders (partners, institutions, funding bodies, donors…).

The information required by each stakeholder should be clearly defined Each stakeholder must define the information that is essential to him/her. This will be done as part of the PME exercise currently underway.

B - What is an indicator ? Objectively verifiable indicator Measurable indicators that show whether the objectives at the three highest levels of the logical framework have been attained. The OVI are the starting point for developing an appropriate monitoring system. UE, 2001

The characteristics of a good indicator A good indicator should be specific – measure what its supposed to measure measurable available at an acceptable cost relevant vis-a-vis a given objective and cover it have an fixed term. UE, 2001

For each indicator, who does what? When? Who ?When ? IndicatorDefines Measures Management (presentation, transmission, archiving…) Decision- making

C – The fields of observation Project / Process Input human, partner, financial, material resources … Environment A project or a process can be schematised as follows : 1st level of results of activities 2nd level of results of activities 3rd level of results of activities Output InputImpact Results

What have we achieved? What stage are we at? Results monitoring What goal(s) have we (not) achieved ?How did we do it? What helps or hinders our work? What are thelong-term repercussions of our work? Process monitoring Context monitoring Impact monitoring The questions to ask: DDC, 1997

Fields of observation matrix Results … planned…unplanned Process …favourable… unfavourable Impact …desired… undesired Context Favourable influencesUnfavourable influences

D - The criteria Producti on factors Project / Process Actual results Expected results Environment Efficacy = actual results / expected results Impact = actual results/ environment Efficiency =actual results /resources (actually) implemented Time Durability = actual result or process / time Relevance = input/ process/ output/ environment Heeren, 2002

To sum up, Results ( service offers, local capacity building …) Process (mobilisation of human resources, financial management…) Context Impact Indicators for assessing the criteria (efficacy and efficiency mainly / impact, relevance and durability to a lesser extent) The stakeholders should together draw up management charts that provide them with a clear vision of exactlywhat stage they are at. The project manager ’s management chart should be based on his/her project’s logical framework, the FDP ’s chart should be based on his/her programme’s strategy.. P M E

Type of information PeriodicityCollectorUtilisationTransmission Campaign reportMonthlyMinistryEfficacy of the programme WHO Governments SurveyAnnualProjectImpactMinistry WHO Who reportTwice a yearMinistry/ ProjectMonitoring production factors WHO (Geneva) Financial reportThree times a yearProject accountantMonitoring production factors WHO Governments Annual reportAnnualProject managerEfficacy of the project WHO Ministry A matrix such as the one below is not only for identifying the indicators, but also for identifying the form the information should take, the periodicity, use and circulation.. A matrix like this is only useful if it remains simple. Exemple de système d’information de gestion pour un projet d’immunisation OMS de Sahelia (In : DW Brinkerhoff & JC Tuthill, 1991)

Tools…

… for collecting information There are many methods for collecting information : –Monitoring sheets, –Individual interviews, –Enquiries, surveys, –Observation methods in the field, –Work meetings –…–…

… for analysing data  Before starting to collect information, it is essential to define what it is to be used for and how it will be processed.  There are many internal and external documents providing analysis tools.  For example, you could refer to the internal document on partnerships for analysing stakeholders ’ strategies.

… for summarising information  At HI we are snowed under with information, not always analysed.  It is essential to provide information that has been analysed and summarised.  One of the main results expected of the PME exercise is in fact the definition of those indicators that are essential to the different stakeholders (FPD, DO,TC, ODP…).

… and always bear in mind : « The quality of monitoring can be measured by the extent to which it facilitates the taking of decisions on project management and implementation. If they do not improve the implementation of the project, any modifications made to the internal monitoring of the project are unjustified. » GTZ

Bibliography The references given here specifically concern monitoring. They are available from the documentation centre or on the internet. Bibliographical references on the PME are given in the general PME presentation available from the Methodology TC unit.

 DDC, July 1997, Monitoring –keeping in touch with reality, 58p.  F3E/ Europact, 2002, monitoring a development project– Approach, systems, indicators, 84p. Available on the F3E website  GTZ, Monitoring in a project, 28p.

END