Monitoring and Evaluation of Service Delivery: Strengthening the Base Dwight Uylett, Principal Director, Standards, Government of Jamaica The whole Public Service will be organized around the needs of its customers, directly accountable to them through guarantees of services which are of the highest quality, accessible, convenient, easy to use, integrated, responsive, cost effective, and which assure redress when things go wrong. Ministry Paper No. 56; Public Sector Modernization Vision and Strategy

A sound Monitoring and Evaluation framework consists of the following base factors : –Current knowledge of where we are. –Structure(s) that will facilitate leadership and requisite changes. –Citizen at the centre. –Awareness of threats to targets and plans to mitigate. –Measure to manage. because a sound M&E framework has to be forward- looking, outcome-oriented, facilitate change and engender service quality!! It is within this context, that well present the Caribbean perspective, looking at successes and learnings.

M&E Emphasis Traditional emphasis placed on base factor Emphasis as dictated by learnings Pitfalls Current knowledge of where we are. High M&E framework focused on what went wrong Structure that will facilitate leadership and requisite changes. Low to MediumHighLeadership seen as a personal thing; not something that can have a structure behind it. Citizen at the centre. LowHighA nice to have, but a structure to really make it happen lacking. Awareness of threats to targets and plans to mitigate. Low to MediumHigh We hope for the best Measure to manage. Low to MediumHighIS/IT acquisitions will solve; just set the performance indicator.

Base Factors What is the current situation? Who will lead? Are the citizens involved and how? What are our threats? Measure to manage.

Desired State: Service Delivery National/ Organizational Goal-Setting; Performance Assessments Social Goal-Setting; Performance Assessments Service Requirements Definition Priority Alignment through Strategic and Corporate Plans Measure & Manage..A Performance Monitoring and Evaluation Unit is being established in the Cabinet Office to facilitate achievement of the desired state.

Actual State: Service Delivery National/ Organizational Goal-Setting; Performance Assessments Social Goal- Setting; Performance Assessments Service Requirements Definition Priority Alignment through Strategic and Corporate Plans Measure & Manage Traditional focus

Base Factors What is the current situation? Who will lead? Are the citizens involved and how? What are our threats? Measure and manage?

The Leader Authority- where does it reside, championing Competence- recruitment, development, retention, rotation, engagement Capacity- tools, opportunities, leveraging Change Agent- the way we work! The Jamaican Model

Base Factors What is the current situation? Who will lead? Are the citizens involved and how? What are our threats? Measure and manage

Citizen Involvement: What we have achieved Citizens Charters- penetration, integrity Impact Assessments- ongoing Social Policy Evaluation – engagement, visibility, integrity – Community Based Policy Monitoring Process Community Based Policy Monitoring Process Consultation policy and mechanism - Consultation Feedback: Lessons to Learn Consultation Feedback: Lessons to Learn

Base Factors What is the current situation? Who will lead? Are the citizens involved and how? What are our threats? Measure and manage

Risk Management An Essential Requirement: A general risk management framework inclusive of identifying current processes, inter- relations with other processes and performance of these processes is an essential requirement. The depth, breadth and importance of a risk management framework usually results in it being owned by a high-level Committee (Board sub-component) although a Risk Manager may be assigned. The Committee approach is also preferable from a Governance perspective as well. Proper Governance practices dictate that you have in place a plan for mitigating high risks!! WE ARE ACTING NOW!

Sample New Disease Risk Matrix Risk FactorNEW DISEASE Sub-Risk Component HIGH RISK (7,5)MEDIUM RISK (3)LOW RISK (1,0)Mitigating Strategy Risk Score Impact Score (7,5,3,1, 0) Service Risk Treatment None available; research low-keyed or no immediate results expected. Research at 'favourable' testing status; secondary treatment capabilities. Treatment exists International collaboration; technical working groups; logistics plan for treatment dissemination ScopeScope not known Scope known only in specified conditions. Scope known in all conditions. Technical working groups and field assessments Growth capacity High; can be spread through common activities and interaction. Medium; can be spread only through localised activities and interaction; community and upwards based. Low. Only through isolated activities and interaction; household based. Demographic profiles, public health assessment and risk profile; identification and monitoring of growth 'sites' Public perception Seen as a non-issue by public generally; no ownership. Focus/interest groups concern and ownership Full national concern and ownership taken. Participatory methodologies; surveys; public debate and information sharing Govt/Health Sector Acknowlegement None.Technical based. Acknowledged; lay person terms. Open Govt./Reporting3 515 This matrix is a high-level summary but much work has to be done in identifying and assessing all components of the matrix. Do not underestimate!

Explaining the Results of the Matrix -Red: Scope and Growth Capacity to be controlled as a first step. -Yellow: Treatment and Govt. Acknowledgement are 2ndary controls/steps esp. if first step implementation flawed. -Green: not an issue now. Risk assessment is dynamic and continuous as Factors change!! The less well managed the risk, the worse the impact. Almost always in Post reviews, obvious factors were not considered that would have prevented or mitigated negative outcomes, hence the risk management concept.

Base Factors What is the current situation? Who will lead? Are the citizens involved and how? What are our threats? Measure and manage.

Measure to Manage Target setting and reporting requirements- [very popular] necessary but not sufficient. –How will we feed the reports? Who will do it? When? Is the data feed in sync with the reporting need? –What are we managing against? Are we managing by personal expertise? gut feelings?

Measure to Manage If we are not able to measure accurately and appropriately then Service Quality will be an elusive concept since we will never converge service consistency and service sufficiency. -The major challenge is appropriately since for service delivery the definition of satisfaction changes frequently. This is quite unlike financial performance measurements, for example, which are fairly static.

Measure to Manage Hitpoints HitpointsThenNowBenefit Data forms enrichment Capture bio-dataCapture performance data too Reduces number of forms to complete, and resources required to monitor each form and bring data together. MOH looking at now to improve real-time health service delivery evaluations. Increase service gateways New physical locations E-serviceIncrease service gateways; reduce duplication; reduce overheads. Customs, Tax Offices, several EAs Reduce access points Multiple stopsOne stopStraight data pull. Currently Tax compliance, lost drivers license replacement and food import/export; strengthen control. Framework approaches Working in Silos; someone has to pull data together; Complex, expensive ERP IT solns. Integration; requires a IT soln. only in the most complex of cases. Increase in agility and cycle time improvements; can get things going faster, consistent information. Customer Service Framework EASYEASY HARDHARD

Base Factors What is the current situation? Who will lead? Are the citizens involved and how? What are our threats? Measure to manage...THANKS FOR ALLOWING ME THE OPPORTUNITY TO SHARE..