The PEFA Indicators – How are they being used Actionable Governance Indicators Course - April 29 th, 2010 Frans Ronsholt PEFA Secretariat

2 Characteristics and roll-out of the PEFA / PFM Performance Measurement Framework What may PEFA Assessments be used for? Content

PEFA indicators characteristics PFM Performance Measurement Framework – 28 performance indicators + 3 donor indicators – Evidence based, rated on 4-point ordinal scale – PFM Performance Report with standard format Launched in June 2005 – multi-agency initiative Application is decentralized – Country teams/stakeholder groups decide if, when and how to implement the assessment work PEFA Secretariat – Neutral body - supports users, monitors application 3

PFM Links to Development Goals PFM system performance Budgetary Outcomes Fiscal / Exp Policy Goals Dev Goals MDGs, PRSP, Political Manifesto Budget deficit, Sector allocations, Investment, Debt ratio, Tax burden etc Fiscal discipline, Strategic allocation, Operational efficiency PEFA Framework Other influencing factors

Evolution of Number of Assessments

6 Regional Coverage of PEFA Assessments April 2010

7 Data coverage –175 assessments done in 110 countries –31 repeat assessments in 27 countries Increasingly used for Sub-National government –27 SN assessments in 14 countries Wide stakeholder involvement – WB & EC leading 85% of assessments, 25 other agencies involved – Government leadership/self-assessment increasing, but not yet norm Public access to reports −Only 56% of final reports in public domain −Links provided at PEFA Framework adoption

Assessment in HICs & MICs - examples OECD/HICs – Norway (national), Switzerland (Canton/state) Large Upper MICs – Brazil, Turkey, Belarus, (Russian Federation), South Africa Large Lower MICs – India, Kazakhstan, Ukraine, Morocco, Egypt, Thailand, Philippines, Indonesia, Colombia, Peru 8

9 What can countries use the PEFA assessments for? Harmonize information needs for all stakeholders Inform PFM reform formulation Monitor results of reform efforts Aid allocation and operational decisions Cross-country comparison

Harmonize information needs Create a shared view of PFM system performance among all major stakeholders at country level – for various purposes Reduce transactions costs of analytical work Provide a starting point for subsequent, collaborative work on PFM reform and capacity building 10

Implement PFM reforms Recommend PFM reform measures Identify main PFM weaknesses High level performance overview Coverage of PFM-PR in Reform Cycle Investigate underlying causes Formulate PFM reform program Identify main PFM weaknesses Recommend PFM reform measures PFM-PR

Inform PFM reform formulation (1) PEFA report is one of several inputs –Identification of main strengths and weaknesses – and potential impact on budgetary outcomes –Other factors: political economy, culture, constitution/legal, resources, capacity at entry Ownership means government decisions on priorities –Government to consider all factors in deciding priorities –Allow ample space for government’s prioritization in the reform dialogue with International Agencies Do not use Indicator scores simplistically –A low score is not sufficient justification for reform –Other factors: relative importance of subject, complexity /timeframe for improvement, interdependence with other elements

Inform PFM reform formulation (2) Complementary analysis to PEFA may be required –Detailed analysis of underlying causes needed for formulation of detailed action plan –Limit such analysis to priority areas –Drill-down tools – some exist, others under development Guidance on using reports for reform formulation – How to establish if a reform program is ‘credible’ – Challenge to develop general approach and toolkit to help government/donor teams identify priorities/sequencing – Such an approach could strengthen country ownership of reform and coordination of donor support – Work in progress

Monitor results of reform efforts Schedule full repeat assessments – every 3-5 years Select a few indicators to monitor more frequently Incorporate into the M&E component of the PFM reform program (Kenya, Zambia) Incorporate as monitoring tool in CAS (Bangladesh) PEFA indicators being used for PFM reform program evaluations (IEG, Multi-donor evaluations) 14

Aid allocation & operational decisions Help to define main system weaknesses & related safeguards for use of country systems PEFA indicators used by many agencies as input to – fiduciary risk assessments (WB/CIFA, DFID, KfW, AFD) – aid allocation instruments (WB/CPIA, NL track record) Do not use PEFA performance ratings as conditionality for disbursements – activity indicators may be more suitable – such measures to be under direct government control 15

Cross-country comparison Regional peer learning events – Eastern Europe, West Africa, Caribbean, Pacific – Suitable for countries that share key characteristics – Often arranged in collaboration with IMF TACs Research – AFTFM/Brookings study identifying regional performance characteristics and reform trends – ODI/di Renzio study – correlating country characteristics to PFM performance 16

17 Issues in country comparison Comparison of two countries must be done very cautiously: - Technical definitions may be different - Need to carefully read each report to understand performance differences behind the scores. - Consider country context, ensure comparison of like with like Comparing the scores alone can be misleading

18 Comparing groups of countries Aggregation requires three decisions –Conversion from ordinal to numerical scale –Weighting of indicators (generally and country specific) –Weighting of countries (for country cluster analysis)

19 Comparing groups of countries No scientifically correct or superior basis for deciding conversion and weights exists –Each user takes those decisions on individual opinion –PEFA program does not endorse any particular method In case aggregation is desired: –Be transparent on aggregation methods used –Discuss reasons for choice –Sensitivity analysis to illustrate impact on findings

Discussion