1. Module 9 Monitoring and Evaluation Tuesday, Oct 14, 2013 Ngo Thi Loan and John Carter

2. Agenda Determining an effective strategy for monitoring and evaluating unemployment protection programs; Planning a periodical actuarial review; Quality monitoring programs for national and local offices; Defining and establishing key performance indicators (KPIs); Development of content and timeliness of monitoring reports; Practice Session: Mechanisms to collect feedback; Practice Session: Identifying KPIs and setting targets to measure the unemployment protection program.

3. Determining an Effective Strategy Need to monitor an unemployment protection program: – Good governance (ILO Convention 102 article 71: state has responsibility to ensure sound administration and delivery of benefits); – Assess whether targets are achievable; – Propose corrective measures when targets are not achieved; Governments acknowledge need to focus on measuring progress toward the attainment of programme objectives and targets;

4. Determining an Effective Strategy (continued) Define main objectives of the unemployment protection programme; Utilize measuring tools such as key performance indicators; Identify performance targets for the programme; Introduce corrective action when targets not achieved; Introduce quality monitoring programs to continually track the quality of the service delivery of the unemployment protection programme.

5. Key Performance Indicators (KPIs) KPIs help organizations to understand how well they are performing in relation to their strategic goals and objectives; They reduce the complex nature of organizational performance to a small number of key indicators in order to make performance more understandable and digestible to us.

6. Key Performance Indicators (KPIs) (continued) All aspects of the unemployment protection programmes can be measured including: Coverage (legal and effective); Adequacy of benefits; Operational matters; Financial matters; Beneficiaries satisfaction.

7. Key Performance Indicators (KPIs) (continued) 1) Coverage: Legal: refer to the percentage of the economically active population that is legally protected; Effective: measured in terms of the number of unemployed receiving unemployment benefits and compared to the total of unemployed; In countries categorized as “high income’, only around 70% of the economically active population is legally covered while only around 40% of those unemployed will receive unemployment benefits.

8. Key Performance Indicators (KPIs) (continued) 2) Adequacy of benefits: Ensure adequate income maintenance for covered workers during periods of unemployment; Are the unemployment benefits adequate enough to assist the unemployed worker and his/her family have sufficient funds for daily living and avoid poverty but not overly generous to encourage “moral hazard?”

9. Key Performance Indicators (KPIs) (continued) 3) All aspects of the process can be measured to some degree and the challenge is to highlight those elements that best reflect the main objectives of the UB programme; “Main objective: to make the correct payment to the correct person at the correct time”; Some important examples of operational KPIs: – Speed of Service: in processing claims for benefit and also payment of benefit; – Accuracy of Benefits: results of on-going quality monitoring programmes to determine the accuracy of payment or the quality of decision making; – Percentage of employers failing to remit contributions on time; – Appeal process: decisions upheld by an appeal board or decisions in favour of the appellant.

10. Key Performance Indicators (KPIs) (continued) Financial matters: Status of UI fun d with full details on revenues and spending as well as the sustainability of the fund; Ratio of these who pay consistently on time v.s. the overall number of contributors.

11. Key Performance Indicators (KPIs) (continued) Satisfaction of Clients: Client satisfaction surveys are effective tools in measuring to what degree insured persons, employers or beneficiaries are satisfied with the services provided by the responsible departments.

12. Key Performance Indicators (KPIs) (continued) KPIs are initially used to evaluate the programme’s overall performance; can also be used to measure individual departmental levels (e.g. Provincial/state or local) to see if they are meeting the specific targets; Setting KPIs: Handout 8-1: SMART approach; distribute to participants;

13. Key Performance Indicators (KPIs) (continued) Avoid the following when determining KPIs: Setting KPIs on the basis that it will be easy to reach certain goals; Selective KPIs can be made public; others could be “secondary KPIs are kept within the department management team; KPIs that are too complex, difficult to understand or to explain; Can be manipulated; Too many KPIs which would become too numerous to allow decision makers to focus on what’s important.

14. Quality Monitoring Programmes For Accuracy of Payment: Suggest a programme such as: – “UI Payment Accuracy Review – UI-PAAR”; – UI Processing Accuracy Review – UI-PRAR”; IssueHCMCDanangHa NoiAll Total claims monitored442439107 Cases with “pay effect errors” 0134 Clerical Errors2439 Compliance42193394 % compliance95%75%85%88%

15. Feedback Purpose of collecting feedback from various stakeholders: – “The best way of finding out about people’s view on any topic is to ask them, simply and clearly, to give their opinion";