1. Monitoring and Evaluation in relation to Performance Management in Government
Oliver Seale and Botshabelo Maja
Public Administration Leadership and Management Academy (Palama)

2. Welcome and expectations
Please introduce yourself and briefly tell us about:
What you do
What you enjoy most in your professional/personal life
A major challenge in your professional/personal life
Please tell us about your expectations for this workshop.
Write down these expectations on the pieces of paper provided
When you have finished, we will group these expectations together into categories and then discuss them.

3. Workshop Themes Part One: Oliver Seale (Slides 1-18)
The cross-cutting nature of monitoring and evaluation (M&E)
Dimensions and definitions of performance management
Strategic planning and its links to performance management and M&E
The introduction of performance management in South Africa
Part Two: Botshabelo Maja (Slides 19-33)
M&E as part of performance management
Revision of basic concepts and definitions for M&E
Purpose of performance management
Elements of performance management
Some steps to be taken for planning performance management
Log frame analysis
Part Three: Oliver Seale (Slides 34-44)
A performance management system
Requirements of a performance management system in government
Role players in a performance management system
Part Four: Botshabelo Maja (Slides 44-48)
Building Batho Pele principles into a performance management system
Summary

4. Conceptual framework for M&E
Description
Existing data bases
Data collection methods
Baseline data/ new data
What will be done (strategy)
Why will it be done (policy)
How will it be done (operations)
Indicators and criteria (how to measure)
When (timeframes)
Monitoring
Existing situation
New project or programme
(sit. analysis)
Planning
System to be used
Indicators
Methods
Baseline data
Inputs
Tracking
Processes
Activities
Problem identification
Feedback to manager
Interventions and modifications
Outputs
Evaluation
Criteria
Assessment
Process
Outcome/Impact
Lessons learned
Feedback
Sustainability
Transferability
Conceptual framework for M&E

5. M&E: Training materials (1)
M&E: MANAGEMENT AND OVERSIGHT COURSES
Basic one-day course for overseers, SMS and other managers and users
M&E and strategic planning
M&E and performance management

6. M&E: Training materials (2)
M&E: COMPETENCY COURSES
M&E three-day orientation course
Indicators for M&E
M&E and data quality
M+E; data-base development and mining
M&E, and quantitative research
M&E and qualitative research
M&E and data analysis
M&E and report-writing
M&E and the legal framework

7. Why M&E is important (1) PROJECT 1
Hand-pump tube-wells for obtaining drinking water from underground sources in Bangladesh
Intended for domestic use for safe drinking water
Initially hailed as a great success
But after 20 years:
Water table dropped because sources were over-pumped for agriculture, irrigation and industrial use
Arsenic contaminated the groundwater
Project now harshly criticised
Source: Black, M. (2007). The no-nonsense guide to international development. Oxford: New Internationalist

8. Why M&E is important (2) PROJECT 2
Small earthen embankments in Rajasthan India
Built in the beds of seasonal rivers to arrest water during monsoon
Made use of local knowledge and natural resources
Initially not recognised internationally
Water harvesting was an important outcome
Associated with community regeneration
Water tables rose
Sufficient water in times of drought
Successful project
Source: Black, M. (2007). The no-nonsense guide to international development. Oxford: New Internationalist

9. Monitoring and evaluation (M&E) as a tool for public sector management
M&E cuts across key management functions within the public sector
M&E is a tool to ensure that work done in government is based on aims and measurable results.
M&E is essential for results-based management system.
M&E findings should be taken into account to successfully carry out various key management functions, including:
Policy;
Strategy;
Finance;
Programme and project; and
Performance (people and organisational)
This workshop focuses specifically on performance management and how M&E can be used as basic tools to support this method of management

10. Why is performance management necessary?
“In spite of having achieved democratic governance for the country, South Africans continue to experience underdevelopment, poverty, infrastructure backlogs and inequitable access to basic services.”[1]
To respond to these inequities, government put in place mechanisms and measures to turn situation around.
One of these measures is a performance management system.
Each national and provincial department and each local authority is required by law to DEVELOP A SYSTEM TO MANAGE ITS PERFORMANCE.
[1] Buffalo City , (2002) : The Integrated Development Plan ,Chapter 22:: Performance Management Framework

11. Dimensions and definitions of performance management
The City of Johannesburg: performance management “the setting and measurement of desired outcomes and activities of an organisation”.
At least two dimensions of performance management
Organisational performance management
Individual performance management
This workshop looks at the links between organisational and individual performance management, but the main focus is on organisational performance management.

12. Individual and organisational performance
Source: “Performance measurement and management”. DPSA presentation to
Parliamentary Portfolio Committee on Public Service and Administration; 12 March 2008.

13. Definition of organisational performance
The Department of Public Service and Administration (DPSA)[1] defines organisational performance:
“the ability of an organisation to use its resources efficiently; and to produce outputs that are consistent with its goals and aims and relevant for its clients and stakeholders”.
But, in addition to outputs, and:
“of equal importance are the outcomes and impact of organisational performance – did the outputs (results) lead to the desired outcomes? Did they have the required impact?”
[1] DPSA “Performance measurement and monitoring”: Presentation to Portfolio Committee, 12 March 2008

14. Strategic planning and its links to performance management and M&E
A strategy is an overall approach and plan for the work to be done by each department or component in all tiers of government for the short (the following year), medium (the following three years) and long term (the following five to ten years).
A strategic plan is a road map to lead an organization from where it is now to where it would like to be in five or ten years’ time.
Strategic planning provides the big picture of what you are doing and where you are going.

15. Preparatory steps for strategic planning
Planning should not take place in a vacuum
You need to know how well you have done in the past for future planning
You need:
The performance contract of the accounting officer with stated aims for the previous year, cascading down to the performance contracts of each individual.
An evaluation of the extent to which each aim has been achieved, in the previous year, including:
Key performance areas and indicators for each aim of the previous year;
The targets for each indicator in the previous year and the extent to which each target has been met;
A more in-depth analysis of each target, in relation processes and activities.
An analysis of the expenditure of the department in relation to these targets.
Armed with this information you can determine:
What needs to be discontinued
What needs to be changed or improved
What needs to be accepted

16. The strategic planning session
Evaluations should form the basis of the annual strategic planning session undertaken in each department.
They should clearly indicate the successes and failures of the past year in relation to the targets, as measured by the indicators, compared to the baseline.
The discussions during the strategic planning session can then focus on setting new priorities and targets based on information of past performance. They can help to identify
what needs to be abandoned,
what needs to be done differently to, or
what needs to be continued, because it has worked successfully.
The new strategic plan should clearly indicate the performance areas, indicators and targets for the following year, and how they are going to be measured, monitored and evaluated.

17. Moving from strategic to operational plans
Strategic and operational planning should be closely linked. If you cannot indicate from the outset how strategic plans are to be implemented, they are in danger of failing.
The operational plan takes the strategic plan further by clearly specifying measurable aims, activities, timelines and responsibilities to achieve the strategic plan and the performance management targets.

18. The introduction of performance management in South Africa
The Public Service Act of 1994 and the Public Service Amendment Act of 2007 devolves powers and duties regarding performance management to each executive authority (EA).
The basic approach of the DPSA[1] to performance management in the three tiers of government in South Africa is the recognition that there is a need to have:
An increased delegation of managerial responsibility and authority to departments and within departments; and
A decentralisation of human resources management to the departments; so that the functions of recruitment, selection and appointment of staff is decentralised, but takes place within a national framework.
[1] DPSA “Performance measurement and monitoring”: Presentation to Portfolio Committee, 12 March 2008

19. Monitoring and evaluation as part of performance management
Clear links between performance management M&E.
All the stages of performance management – from planning through to implementation, outputs, outcomes and impacts – need to include ways in which M&E can be implemented.
Performance management and monitoring, or continually tracking the progress of the work done in a department, directly linked.
Performance management and evaluation are also closely linked.
The cyclical nature of performance management and M&E.

20. Uses of M&E M&E are essential tools to:
Support evidence-based policy-making, particularly in the context of the budget cycle and planning;
Support government departments in managing activities, including service delivery and the management of staff;
Enhance transparency and support accountability, including the accountability of government to parliament, provincial legislatures and civil society; and
Support accountability within government, for example accountability between ministers, managers and staff.

21. Revision of basic concepts and definitions for M&E
Interventions
An intervention is defined as a programme, project or service, implemented at all levels of government
Monitoring
Monitoring involves the tracking of inputs, processes, activities, outputs and outcomes against indicators, and the modification of these processes and activities as and when necessary
Evaluation
Evaluation is a systematic assessment of the strengths and weaknesses of the design, implementation and the results of completed or ongoing interventions. Its aim is to help to improve future interventions
The government-wide M&E system
The Government Wide Monitoring and Evaluation System (GWMES) is an overarching framework for collecting information within government
Indicators
Monitoring and evaluation processes require a set of indicators, which are used to measure the progress of an intervention. Indicators should be measurable, aim, verifiable, and clearly understandable.

22. Purpose of performance management
Performance management is an instrument for each person in the public sector to use, from the accounting officer downwards, to manage their work in relation to achieving the required results of the department.
The overall aim of performance management is to enhance efficiency and effectiveness of a department in relation to improved service delivery.
Monitoring and evaluation are the tracking and appraisal tools to be used to measure progress in attaining these results.
Taken together, performance management and M&E provide the guidelines and standards on which performance can be measured, developed and rewarded.

23. Elements of organisational performance management
At the organisational level, performance management comprises: [1]
Planning
Strategic planning – indicates what you are going to do
Operational planning – indicates how you are going to do it
Implementation
Monitoring system, which focuses on how you know you are on track
Evaluation, which examines how well you have done what you set out to do.
Feedback
New planning taking past performance into account.
[1] Adapted from Tau, M. (2008). “Municipal organizational performance management guide”. North West Province: Department of Local Government and Housing

24. Elements of individual performance management
Setting goals, objectives and targets in alignment with the organisation’s strategic and operational plans
Implementing goals, objectives and targets
Monitoring this implementation
Performance Appraisal
Performance improvement plan
Granting of rewards in line with the DPSA Incentive Policy Framework (2003)

25. Steps towards building a performance management system
A system is a set of components, for example indicators, activities, processes, programmes or projects comprising a total picture
Let us think of a system for primary health care in a specific clinic. Such a system would consist of the various components, for example:
A method of working
A method of record-keeping
A directory of the services that the clinic offers
The tasks and responsibilities of each staff member of the clinic
The protocols to be followed when offering health care
The services that the clinic offers to the community outside of the clinic
The support and maintenance services that the clinic requires
The budget attached to each component of the work
This is not yet a system since the components are not linked

26. Please complete Exercise 1

27. Log frame analysis
A log frame is a matrix table, which in turn is a tool for planning and managing work.
It can turn components, indicators, activities etc. into a system by linking them to each other
It aims to present information about the key components of work in a clear, concise, logical and systematic way. It summarises, in a standard format:
What the work is going to achieve?
What activities will be carried out to achieve its outputs and purpose?
What resources (inputs) are required?
What are the potential problems which could affect the success of the work?
How the progress and ultimate success of the work will be measured and verified?

28. Elements of the logical framework approach
Situation analysis
Stakeholder or participation analysis
Problem Analysis
Aims Analysis
Alternatives Analysis
Activities Planning 

29. A log frame matrix table
A log frame matrix table is the actual result of the log frame analysis.
There is a logical connection between the cells of the log frame matrix.
The logic that connects the cells in the left-most column is referred to as the vertical logic. It is the hierarchy of aims of the project.
The logic moves from goal, through to purpose, outputs and activities.
The logic that connects the rows through the remaining three columns is referred to as the horizontal logic. For a given level of aim (for example at the level of “purpose”) the horizontal logic describes:
How the achievement of the aim will be measured or verified;
How this information will be obtained; and
The external factors that could prevent the project manager and staff from achieving the next level aim.

30. Log frames
Aims
Measurable
Indicators
Means of
Verification
Important
Assumptions
GOAL (DESIRED IMPACT):
(Goal to super goal)
PURPOSE (DESIRED OUTCOME):
(Purpose to goal or impact)
OUTPUTS:
(Outputs to purpose or outcome)
ACTIVITIES:
INPUTS:
(Activity to output)

31. Aims Measurable Indicators Means of Verification Important Assumptions
GOAL (DESIRED IMPACT):
Wider problem the
project will help to
resolve
Quantitative ways of
measuring or
qualitative ways of
assessing achievement of goal
Cost-effective
methods and
sources to quantify
or assess indicators
(Goal to super goal)
External factors
necessary to sustain
aims in the long
run
PURPOSE (DESIRED OUTCOME):
The immediate impact
on the project area or
target group i.e. the
change or benefit to
be achieved by the
project
assessing achievement of
purpose or desired outcomes
(Purpose to goal or impact)
External conditions
necessary if achieved
project purpose is to
contribute to reaching
project goal
OUTPUTS:
These are the
specifically deliverable
results expected from
the project to attain
the purpose
qualitative ways assessing the production of outputs
(Outputs to purpose or outcome)
Factors out of project
control which, if present,
could restrict progress
from outputs to
achieving project
purpose
ACTIVITIES:
These are the tasks to
be done to produce
the outputs
INPUTS:
This is a summary of
the financial and non-financial resources required to do the work
Financial and non-financial monthly, quarterly and annual reports
(Activity to output)
Factors that cannot be controlled, which, if present, could restrict progress from activities to achieving outputs

32. Completing a log frame Aims Measurable Indicators Means of
Verification
Important
Assumptions
GOAL (DESIRED IMPACT):
Eliminate polio among children in the local authority
Number of children in a local authority who have contracted polio in the year after the introduction of a polio immunisation programme
Examination of the notifiable diseases register
(Goal to super goal)
Cases of polio are diagnosed and reported
PURPOSE (DESIRED OUTCOME):
Full immunisation against polio
Percentage of children who have received full polio immunisation in a given year
Household survey with examination of road –to-health cards
(Purpose to goal or impact)
Household members will retain road-to-health cards
OUTPUTS:
Immunisation clinics
Mobile clinics
Immunisation education programmes
Number of clinics
Number of parents or caregivers receiving education about polio
Clinic records
(Outputs to purpose or outcome)
Clinic records actually reflect the programmes
ACTIVITIES:
Staff training
Setting up clinics
Setting up inoculation protocols
Designing education material
INPUTS:
Nurses, community health workers, education materials, mobile clinics, vaccination materials
Financial and non-financial monthly, quarterly and annual reports
(Activity to output)
Record-keeping is accurate
Source: Adapted from

33. Please complete Exercise 2

34. A performance management system
A performance management system is a system of components designed specifically to manage and measure how well a government department or municipality is doing in reaching its desired results.
It includes methods of monitoring and evaluation of each dimension of the system through key indicators.
A performance management system includes both organisational and individual performance measurements.
It needs to indicate how the various components of work, including individual performance, form a coherent picture of the work done in relation to the results achieved.

35. Elements of a performance management system
Governance/ management performance
A performance management system should contain interconnected information on:
Financial performance
Performance in relation to the specific intervention/ project/ programme
Individual performance

36. Building a performance management system
The 10-step approach of Kuzak and Rist (modified)
Step 1: Conducting a situational analysis..
Step 2: Agreeing on performance outcomes
Step 3: Developing key indicators to monitor outcomes.
Step 4: Gathering baseline data on indicators.
Step 5: Planning for improvements – setting realistic targets.
Step 6: Building a monitoring system.
Step 7: Analysing and reporting findings.
Step 8: Collecting and providing evaluative information.
Step 9: Using the findings.
Step 10: Sustaining the M&E system within government. .
Links to log frame approach
Focus on M&E information and its collection, report-back and using the findings.
The cyclical nature and the sustainability of the system is stressed.
Modified from Kuzak, J Z and Rist, R C. (2001) . “Building a performance-based monitoring and evaluation system”. Evaluation Journal of Australasia. 1 (2), pp

37. Developing performance measures
Step 1:
Select appropriate key performance areas
Step 2:
Formulate appropriate aims
Step 3:
Develop appropriate key performance indicators
Step 7:
Set target dates for starting and completion of activities
Step 8:
Allocate financial resources to achieve targets
Communicate with staff and stakeholders
Step 6:
Allocate responsibility for each activity
Step 5:
Identify activities required to achieve performance targets
Step 4:
Set performance targets
Source:: Tau, M. (2008). “Municipal organizational performance management guide”.
North West Province : Department of Local Government and Housing

38. Case study on a performance management system (1)
Step 1: Conducting a situational analysis.
Management of TB and the prevention of its spread is a major concern of the community.
Step 2: Agreeing on performance outcomes
Community requires specific programmes that focus on TB prevention and treatment.
Step 3: Developing key indicators to monitor outcomes
The percentage of people in the community, at a given point in time, who test positive for TB (TB prevalence).
Step 4: Gathering baseline data on indicators.
Community-based household survey, with testing facilities for TB.
Step 5: Planning for improvements – setting realistic targets.
Reduce TB prevalence by 10% from the baseline in the first year of the programme.

39. Case study on a performance management system (2)
Step 6: Building a monitoring system.
Allocate resources (including staff and finances) to each aspect of plan. Monitor each components.
Step 7: Analysing and reporting findings.
Staff and finances allocated to analysing the data in TB monitoring system + reporting to clinic management.
Step 8: Collecting and providing evaluative information.
At the end of the first year repeat the household survey, with TB diagnosis facilities.
Step 9: Using the findings.
Information from the TB monitoring system and the evaluation will feed into the planning cycle for the following year.
Step 10: Sustaining the M&E system
We will ensure that our TB programmes and monitoring systems are sustainable through ongoing measurement and reporting

40. Please complete Exercise 3

41. Requirements of a performance management system in government
Each department or local authority should develop its own performance management system.
The system should focus on the work to be done by the department in relation to its specific goals and the priorities it aims to achieve.
It should link the performance agreement of the accounting officer with her/his political head to the work that each component of the department will do in order fulfil these goals and priorities.
It should therefore link the aims of the department with its operational activities.
The performance of each employee should be linked to these operational activities to ensure increased, efficient and effective service delivery.

42. Role players in a performance management system
Accounting officer:
Each financial year the head of a national or provincial department or a local authority (the accounting officer) signs a performance contract with his/her political head, indicating what the organisation as a whole should achieve within that year.
Senior managers:
The performance contract of the head of the department is cascaded down to the senior managers. Each senior manager, in turn, signs a performance contract with his/her direct manager, agreeing to the achieving the results that are relevant to his/her particular component.
Managers
The managers (deputy directors and assistant directors) are directly responsible for actual operational performance in their units. The accuracy of the monthly, quarterly and annual reports in relation to set targets, are essentially their responsibility.
Supervisors:
The performance contract of the manager of a unit is then cascaded down to the supervisors of sub-units.
Employees:
Each employee should take responsibility for his/her own performance in relation to the specific activities assigned.

44. Performance measurement and management
Performance at organisational level: How well is the organisation doing in attaining its overall goals and objectives?
Performance at the process level: How well is each branch/component/division doing in attaining its specific goals and objectives?
Each of these branches/components/divisions contributes to the bigger picture of how well the organisation is doing.
Performance at individual level: How well is the individual is doing in relation to:
Her/his specific area of work,
The branch/component/division, and
the bigger picture of how well the organisation is doing.

45. Batho Pele, performance management and M&E
Batho Pele principles:
Consultation
Service standards
Access
Courtesy:
Information:
Openness and Transparency:
Redress
Value for money

46. Building Batho Pele principles into a performance management system
Implementation of Batho Pele principles can be measured, monitored and evaluated.
For example, community consultation can be measured both qualitatively and quantitatively.
Interviews can be held with community leaders, focus group discussions can be held with recipients of a service, and/or a community survey can be conducted to examine the extent to which consultation has taken place with the people living in the community.
Key indicators based on the community consultation should be developed, but these may be different for each department.
User satisfaction surveys can indicate the extent of satisfaction with the service, the extent to which expectations were met and the standards of the service.
As part of performance contracts each public sector worker, from the accounting officer and cascading downwards, can be asked to indicate how they are going to practice Batho Pele in their specific work situation.

47. Summary The cross-cutting nature of monitoring and evaluation (M&E)
Dimensions and definitions of performance management
Strategic planning and its links to performance management and M&E
The introduction of performance management in South Africa
Monitoring and evaluation as part of performance management
Revision of basic concepts and definitions for M&E
Purpose of performance management
Elements of performance management
Some steps to be taken for planning performance management
Log frame analysis
A performance management system
Requirements of a performance management system in government
Role players in a performance management system
Building Batho Pele principles into a performance management system

48. Thank you