The PEFA Program – and the PFM Performance Measurement Framework Public Financial Analysis and Management (PFAM) Course World Bank Washington DC, April 24, 2007 Frans Ronsholt PEFA Secretariat

Content What is PEFA ? The Strengthened Approach to Supporting PFM Reform The PFM Performance Measurement Framework Roll-out of the Framework Assessment Process Issues The Role of the PEFA Secretariat

What is PEFA ? Public Expenditure and Financial Accountability Program aimed at harmonization and alignment supporting the Monterrey, Rome and Paris Declarations Established by a core group of international financial institutions and donor agencies World Bank, IMF, European Commission, UK, France, Norway, Switzerland Guides and finances the Program Working closely with other donor agencies through the OECD-DAC Joint Venture on PFM PEFA Secretariat located within World Bank The PEFA Program was established in 2001 The Secretariat was located within the World Bank offices in order to give it a legal/contractual identity and because the Bank offers the most extensive technical capacity in PFM among the PEFA partners and in that way supports the Secretariat’s work. Transition to next slide: Why was the PEFA program established?

PFM Diagnostics in the 1990s Large amount of PFM work undertaken, mostly by development agencies a good deal of knowledge generated. LIMITATIONS Duplication and lack of coordination led to heavy burden on partner governments. Not possible to demonstrate improvements in PFM performance over time in a country Monitoring of PFM reforms focused on inputs and activities, rather than performance The PFM work was used mainly for two purposes: To guide PFM reform preparation To feed into the fiduciary risk assessments of individual donor/financing agencies

The Strengthened Approach to Supporting PFM Reform A country-led PFM reform program including a strategy and action plan reflecting country priorities; implemented through government structures A donor coordinated program of support covering analytical, technical and financial support A common information pool based on a framework for measuring and monitoring results over time i.e. the PEFA PFM Performance Measurement Framework This approach was adopted by the Bank through the memorandum issued to all sectors jointly by the Vice Presidents of PREM and OPCS in July 2005

The PFM Performance Measurement Framework The PEFA Framework is publicly available to be used by any interested party. (Handout of booklets and sheet with indicator list and scoring example)

Components of the Framework A standard set of high level PFM indicators to assess performance 28 government performance indicators 3 donor indicators, reflecting donor practices influencing the government’s PFM A concise, integrated report – the PFM Performance Report Standard content and format provides the narrative to support the indicator assessments (the evidence) draws a summary from the analysis

Coverage of the Public Sector Focused on central government operations Links to other parts of the public sector Sub-National Governments Public Business Enterprises to the extent these have implications for Central Government May be applied to sub-national government - Requires minor modifications

Principles of Indicator Design High level system performance is measured Assesses performance, but not underlying capacity factors Full overview of the PFM system revenue, expenditure, procurement, financial assets/ liabilities Basis for design: The 16 HIPC Expenditure Tracking Indicators, but broader draws on IMF’s Fiscal Standards and Codes (ROSC) internationally accepted standards e.g. GFS, IPSAS, INTOSAI Widely applicable to countries at all levels of development, but not intended for cross-country comparison The indicators measure SYSTEMS, and NOT fiscal or expenditure POLICY It measures PERFORMANCE, NOT underlying CAPACITY FACTORS such as legislation, organizational structures, IT systems, staff numbers and qualifications. No direct reference to ‘poverty reduction’ which is an issue or a policy concept in many but not all countries.

Structure of the Indicator Set C. Budget Cycle Policy Based budgeting D. Donor Practices A. PFM Out-turns B. Cross-cutting features External scrutiny and audit Predictability and control in Budget Execution Budget credibility Budget credibility indicators reflect outputs of the system as a result of the institutional and more qualitative assessment covered by the remainder of the indicator set. Understanding links between the institutional strengths/weaknesses and budget credibility is important for summarizing the assessment. Similar for donor indicator ratings’ relations to government performance and budget credibility. Comprehensiveness and Transparency Accounting, Recording, Reporting

Content of Indicator Set A. PFM Out-turns Credibility of the budget Indicators 1- 4 Deviations from aggregate budgeted expenditure and revenue as well as expenditure composition. Level of expenditure arrears. B. Key Cross-cutting issues Comprehensiveness and transparency Indicators 5-10 Coverage of budget classification, budget documentation, reporting on extra-budgetary operations, inter-governmental fiscal relations, fiscal risk oversight and public access to information. C. Budget Cycle i. Policy-based budgeting Indicators 11-12 Annual budget preparation process, multi-year perspective in fiscal planning, expenditure policy and budgeting Examples of tricky indicators: PI-4 : definition of arrears from an economic rather than accounting point of view. PI-7 : How to measure what is not included in fiscal reports. Triangulation from many sources.

Content of Indicator Set (cont’d) C. Budget Cycle ii. Predictability & control in budget execution Indicators 13-21 Revenue administration, predictability in availability of funds, cash balances, debt & guarantee management, payroll controls, procurement, internal controls and internal audit iii. Accounting, recording and reporting Indicators 22-25 Accounts reconciliation, reporting on resources at service outlet level, in-year budget execution reports, financial statements iv. External scrutiny and audit Indicators 26-28 Scope, nature and follow-up on external audit; legislative scrutiny of annual budget law and external audit reports D. Donor Practices Indicators D1- D3 Predictability of direct budget support; donor information for budgeting and reporting; use of national procedures Procurement incorporated in many indicators, not just covered by PI-19 (e.g. PI-4, PI-12, PI-16, PI-20, PI-26)

Calibration and Scoring Calibrated on four point ordinal scale (A, B, C, D) Requirements for each score explicitly specified Scoring based on extent of internationally recognized ‘Good Practice’ Indicators have 1, 2, 3 or 4 dimensions in total 74 dimensions to provide detailed information & transparency of score each dimension must be rated separately Aggregation only from dimensions to indicator Aggregation of indicator scores is not encouraged because: An overall score (number) does not assist in identifying strengths and weaknesses to guide reform dialogue on priorities and more detailed analytical work needed; such an aggregate mainly serves country comparison which is not the intention of the Framework; standard aggregation assumes that all indicators should be given the same weight in all countries which is not the case (e.g. importance of SN government, SOEs and external aid) aggregates mainly serve for individual donor decisions on allocation of aid and modalities of aid implementation. Different agencies assign different weight to the individual parts of the PFM system and therefore use different weights.

Roll-out of PEFA based Assessments

Roll-out of PFM Assessments PFM Performance Measurement Framework launched June 2005 Assessment Status as at March 2007 45 substantially completed i.e. draft/final report 24 on-going but report not yet issued 27 agreed with government but not started Roll-out rate: a steady 2-3 new assessments per month Outlook for mid 2008 75-80 countries covered 8-10 repeat assessments

Geographical distribution

Assessment Process Issues

Decentralized Process Application of the PEFA Framework to be decided at country level. Decisions to be made: If and Why ? When ? How ? Recommended by international organizations as good practice (e.g. OECD-DAC, ComSec) No supra-agency mandates or responsibilities. Each country and organization decides its interest in a PEFA assessment and ability to contribute.

Government Involvement Government’s role Self-assessment (with external validation) Joint assessment (joint team) Collaboration with donor-led assessment Determined by interest and capacity What are the benefits to government? Government staff may need training

Donor Collaboration A donor reference group is essential to ensure that needs of all parties are addressed to ensure common acceptance of findings The reference group to agree internally and with the government on: Diagnostic packaging Resources for assessment work Stages and timing of the assessment work Quality assurance arrangements

Diagnostic Packaging - Purposes of Standard Diagnostic Tools The matrix brings together and illustrates the purpose for which each of the diagnostic tools is applied. You will notice that it is only the PFM report which gives the high level overview of PFM performance The PFM-PR does not undertake an in-depth analysis of the capacity factors, nor provide recommendations for reform.

Diagnostic Packaging – Coverage of PFM Performance Report Implement PFM reforms High level performance overview Formulate PFM reform program PFM-PR Before embarking on the formulation of a reform program, it is necessary to obtain a high level overview of the system performance and aclear understanding of the main PFM weaknesses. The causes of these weaknesses need to be investigated and this may entail the application of further diagnostic work to drill down into the areas of concern. The recommendations arising from these investigations will then be used to formulate a PFM reform program. Identify main PFM weaknesses Recommend PFM reform measures Identify main PFM weaknesses Recommend PFM reform measures Investigate underlying causes

Diagnostic Packages - a problematic concept; rarely working well Stand-alone PFM-PR - Recommended -e.g. input to decision on focus of subsequent in-depth analysis PFM Performance indicators integrated into other product - in combination with PER, CFAA or PEMFAR/CIFA - a problematic concept; rarely working well PFM-PR as a separate/early module of a broader analytical product - often works well (e.g. Afghanistan PER, Ghana ERPFM) Reports may be classified into three groups according to whether the reports (i) are ‘stand-alone’ reports that essentially follow the structure and content of a PFM Performance Report (PR), or (ii) follow the structure and content of a different analytical product with the indicator-assessment incorporated, or (iii) constitutes a PFM-PR type report as a separate annex, volume or report of a broader analytical product i.e. a ‘dual’ product. With the stand alone reports they are usually issued as consultants reports ( with the exception of Zambia). In the second scenario where the PFM performance indicators are integrated within the other product the report is issued as that of the Institution e.g. World Bank products. The third scenario where you have dual products does not follow any specific pattern and the reports are issued as either consultants’ or donor agency reports as arrangements may have determined. Fiduciary risk assessments tend to be donor specific, and there are distinct advantages of seperating a FRA from a PFM-PR.

Quality Assurance Terms of Reference & Draft Report to be Q.A.’d Government and donor reference group Should ensure that information is used correctly and reflects the situation in the country Donor specific arrangements to be respected e.g. World Bank peer review mechanism The PEFA Secretariat can contribute as peer reviewer, if requested Will consider if the product respects the Framework’s standards and methodology In-country assessment reference groups (typically including the government and donors beyond those directly participating in the assessment team) provide an important review mechanism in that they should have the best impression of the quality and completeness of the information provided in the reports and the extent to which appropriate professional judgment has been exercised in areas of limited data availability. Such arrangements have been made in Kyrgyz, Moldova, Uganda and Mozambique. A weakness of a very comprehensive team of donor agency staff is that few, if any, informed donor representatives will be left in-country to take an arms-length view of the quality of the assessment (Tanzania). Some donors have specific quality assurance arrangements which their teams will have to also comply with. For example the World Bank’s institutionalized peer review system. The PEFA Secretariat’s contribution to the peer review process is mainly to offer an opinion on the extent to which the Framework’s assessment scope, methods and report provisions have been met. The Secretariat usually cannot judge the accuracy and completeness of the data used for the exercise, but does assess adequacy of data presented to justify the scoring.

The PEFA Secretariat

Mission of the PEFA Secretariat To disseminate information on the Strengthened Approach to Supporting PFM Reform and the PEFA Performance Measurement Framework. To support applications of the PEFA Framework at country level for quality and usefulness of the assessments. Reports to the seven-agency PEFA Steering Committee The PEFA Program and Secretariat do NOT initiate, undertake or finance the assessments.

Activities of the PEFA Secretariat Development of the Framework; interpretation and clarification of the indicators Provision of support and guidance for quality in implementing the assessment in each country Development of training programs and materials Monitoring roll-out of the strengthened approach for lesson learning and input into training and guidance

Support Services and Tools Support tools for assessment managers on the website (www.pefa.org): List of completed, ongoing and planned assessments – updated 3-4 times annually Links to completed reports, if they are made public Support to assessment managers on request: Advice / Video-conference briefings to country teams on assessment planning List of consultants with PEFA assessment experience Review of terms of reference Quality reviews of draft assessment reports

Support Services and Tools Support tools for assessors & trainers on the website: Translations of the Framework (English, French, Spanish, Portuguese, Russian, Arabic) Calculation spreadsheets for some indicators Guidance on information / evidence for assessment Clarifications and additional guidance on indicators Training materials Support to assessors on request: Indicator interpretation and other advice to assessors during implementation

Thank You for Your Attention