1. The PEFA Program - an Overview
Presentation to be used by SC members
This presentation has been prepared by the Secretariat for use by SC members.
It gives an overview of the PEFA program, PEFA Framework and PEFA Secretariat input.
It may be shortened or more details may be needed on specific issues depending on audience and available time.
Prepared by the PEFA Secretariat – June 2011

2. Content - Program Overview - The PEFA Framework - Use & Impact - The PEFA Secretariat
This section presents:
The Program
its stakeholders
the Strengthened Approach (3 pillars0
The components
The 6 critical dimensions of the FW
Purpose of the FW
Implementation modalities and typical stage of an assessment

3. Program Overview This section presents: The Program its stakeholders
the Strengthened Approach (3 pillars0

4. What is the PEFA program?
PEFA = Public Expenditure & Financial Accountability Program
Objective:
Results orientation in development of PFM systems
Harmonization of PFM analytical work
Established:
Established in 2001 by seven agencies.
Working in tandem with the OECD-DAC Task Force on PFM
Contributing to development effectiveness through: The Strengthened Approach to Supporting PFM Reform
Objectives: based on the realization that former diagnostic tools did not enable measurement of change over time, led to conflicting conclusions and to duplication of efforts by individual development agencies with large transactions cost for the partner government
Seven agencies: listed in next slide [#5] (diagram)
These seven agencies constitute the PEFA Steering Committee. Committee meets semi-annually with rotating chair.
OECD-DAC Task Force on PFM (formerly: Joint Venture on PFM): Set up in relation to the Paris Declaration and subsequent Accra Agenda for Action. Linked to Working Party on Aid Effectiveness – the Country Systems Cluster
The Strengthened Approach : more details in slide 6

5. The PEFA stakeholders
7 partners: 3 international agencies and 4 countries

6. The Strengthened Approach to Supporting PFM Reform
A country-led PFM reform program
including a strategy and action plan reflecting country priorities; implemented through government structures
A donor coordinated program of support
covering analytical, technical and financial support
A common information pool
based on a framework for measuring performance and monitoring results over time
i.e. the PFM Performance Measurement Framework
Strengthened Approach developed in parallel to work leading to the Paris Declaration Is completely consistent with the PD but focused only on PFM aspects.

7. ThE PFM PERFORMANCE MEASUREMENT FRAMEWORK
This section presents:
The purpose
The components
The 6 critical dimensions of the FW
Implementation modalities and typical stages of an assessment

8. The PEFA Framework PFM Performance Measurement Framework
Better known as ‘the PEFA Framework’
The ‘flagship’ product of the PEFA Program
Launched in June 2005
Designed to measure performance of national PFM systems
Applicable to countries and different stages of development
Also applicable for Sub-National government systems
The framework was developed
Exposure draft February 2004.
Was tested in 25 countries by means of desk and live assessments.
Wide consultation
Special guidance available for use at sub-national level

9. PFM Links to Development Goals
PFM system performance
Budgetary Outcomes
Fiscal/Exp Policy Goals
Service Delivery Goals
Dev Goals
MDGs, PRSP, Political Manifesto
Budget deficit, Sector allocations, Investment, Debt ratio, Tax burden etc
Fiscal discipline, Strategic allocation, Operational efficiency
PEFA Framework
Other influencing factors
PFM systems enable governments to deliver on the budgetary outcomes, and in turn the fiscal/expenditure policy targets determined by political processes.
These targets are important contributors to achieving the service delivery targets and eventually the high level development outcomes as expressed in the Political Manifestos, Millennium Development Goals, and Poverty Reduction Strategies.
Other influencing factors that ultimate influence the achievement of goals include external shocks (such natural disasters, international commodity prices or refugee influx)

10. Purpose of the PEFA Framework
Objective
To determine if a country has the tools to deliver three main budgetary outcomes (aggregate fiscal discipline; strategic resource allocation; efficient service delivery)?
It provides:
a high level overview of all aspects of a country’s PFM systems performance
includes revenue, expenditure, procurement, financial assets/ liabilities
It does not provide:
Assessment of underlying causes for good / poor performance
An assessment of government fiscal/expenditure policies
3 main budgetary outcomes are:
Effective controls of the budget aggregates and management of fiscal risks contribute to maintain aggregate fiscal discipline.
Planning and executing the budget in line with government priorities contributes to implementation of government’s objectives (strategic resource allocation).
Operational efficiency in managing the use of public resources contributes to efficient service delivery and value for money.
[cf. Blue Book p.1]
The Framework measures performance by looking at the outputs from the PFM systems, but not the reasons behind performance such as institutional, organizational and human resource aspects of capacity. [It can be considered as an annual medical with your GP. It will point to weaknesses, causes of which are easy to identify and remidy in some cases, whereas in other cases investigation by specialists will be needed]
The Framework does not assess the government’s fiscal and expenditure policies, but only the performance of the systems which the government needs in order to implement such policies.
It is intended to be widely applicable across countries with different levels of economic development.

11. Components of the PEFA Framework
A standard set of high level PFM performance indicators (PIs)
31 performance indicators (with 76 sub-indicators)
Grouped in six critical dimensions
A compromise between simplicity and comprehensiveness
A standard report format
the PFM Performance Report (PFM-PR)
developed to provide country background, evidence on the indicators and an integrated, analytical summary.
The set of indicators includes:
28 Government PIs and + 3 Donors PIs. Each indicator may have 1-4 sub-indicators (or indicator dimensions)
Indicators (and their dimensions) are scored on a 4-grade ordinal scale (A, B, C, D)
The limited number of indicators make the assessments manageable in terms of resource demand/cost (average cost $125,000 per assessment)
Donor indicators add an element of mutual accountability. Are not applicable in countries with negligible aid receipts.
An additional indicator ‘HLG1’ required for sub national level government assessments
The analytical summary provides the story line – based on the information in chapters 1-4.

12. Dimensions of PFM system performance
Policy-based budgeting:
Is the budget prepared with due regard to government policy?
Accounting, recording & reporting:
Are adequate records & information produced, maintained & disseminated to meet decision-making, control, management & reporting purposes?
Comprehensiveness and transparency :
Are the budget & fiscal risk oversight comprehensive, & is fiscal & budget information accessible to the public?
Budget credibility:
Is the budget realistic, & implemented as intended?
Predictability & control in budget execution:
Is the budget implemented in a predictable manner & is control & stewardship exercised in the collection & use of public funds?
External scrutiny & audit:
Are there effective arrangements for scrutiny of public finances & follow up by the executive?
6 critical dimensions of PFM system performance
The assessment shall look at each of the six dimensions in turn in relation to country context – It shall also look at how the six dimensions relate to each other.
There are many ways to slice a cake and the same goes for PFM – the PEFA framework is based on many years of development and also extensive field trials – there is therefore a high degree of confidence that the indicator set captures the key dimensions of PFM performance.
This organizing framework is used both for the performance indicator set and for the first level of summarizing the indicator assessments.
Policy based budgeting – includes the involvement of the political and sector management levels in the budgeting process.
Budget credibility is a result of the performance of the PFM systems i.e. performance on indicators #1-#4 is largely a function of indicators #5-#28 (or #5-31)

13. Implementation modalities
Country focus and decision
Application of the PEFA Framework is entirely decentralized to the country level (if, when, how to use Framework)
Inclusiveness
All stakeholders can be involved and any agency can in principle undertake any role in its implementation (managing, financing, quality review)
The Framework does not ‘belong’ to any single organization
Due process is crucial to acceptance and use - Joint work is encouraged
The PEFA tool appears as a ‘common good’ – available to any interested party
There is no central planning or financing of PEFA asessments

14. Stages in a typical process
0. Agree the intention to undertake a PEFA based assessment
1. Agree purpose, scope and stakeholder roles
2. Prepare TOR
3. Mobilize assessment team
4. Introduction workshop for stakeholders
5. Review of existing information
6. Inception Report
7. Main field work
8. 1st Draft Report
9. Quality Review
10. Supplementary field work
11. Draft Final Report
12. Presentation seminar
13. Final report
14. Use of the report for reform dialogue
The sheer number of steps in a typical process (in this simplified presentation)
There is no “template” - each assessment should be driven by its own needs and requirements. This list illustrates important steps that should be considered in any assessment – but not be seen as a blueprint.
This process does not include a government self-assessment and report, which may replace or run parallel to steps 5 and 6.
The assessment team (consultants) may be needed in–country on 4 occasions (steps 4, 7, 10, 12). This is in practice - unfortunately - often cut down in order to save on the budget.
It may be useful to arrange a validation workshop with government as introduction to the supplementary field mission, step 10.

15. USE & IMPACT of the pefa framework

16. The early assessments took place in Africa and the Pacific
The early assessments took place in Africa and the Pacific. Many Insular countries in the Pacific or the Caribbean also. Has picked up well in Latin America, Eastern Europe and Central/South Asia. Use is spreading at a slower rate in the Middle East and East Asia.
Many sub-national assessments have taken place – especially in large federal states such as Brazil, Argentina, Nigeria, Ethiopia, Pakistan, India and Switzerland. But also municipalities such a Bogota, Dakar, Ouagadougou, Tblisi.

17. PEFA Framework adoption
Very good progress – globally
220+ assessments substantially completed, covering 120+ countries.
Since 2010, mostly Repeat assessments and Sub-National assessments
High country coverage in many regions
Africa and Caribbean 90% of countries
Latin America, Eastern Europe, Asia, Pacific 50-80%
Used in many Middle Income Countries
Upper MICs: Brazil, Turkey, Belarus, Russian Federation, South Africa
Lower MICs: India, Kazakhstan, Ukraine, Morocco, Egypt, Thailand, Philippines, Indonesia, Colombia, Peru
Repeat assessments typically take place after a period of 3 years from the baseline. Some countries have completed 3 comparative PEFA assessments.
Most assessment have taken place in low income countries; but many MICs are also applying the Framework, as do a few HICs.
HICs: Norway, Switzerland, Kuwait
Large MICs: South Africa, Brazil, India, Ukraine, Morocco,, Philippines, Indonesia, Peru are final and publicly available, as are India (Himachal Pradesh)
Thailand, Turkey and Kazakhstan are final but non public.
Russian federation is in draft version but a final version will not be prepared.

18. Stakeholder involvement
Government leadership/self-assessment
Active government participation in assessment is key to the use in reform policy dialogue
Full self-assessment may be very demanding on govt. capacity
Increasing frequency of Self-assessment with expert assistance and external validation – but still a minority of cases
Development Agencies
About 25 development banks and donor agencies involved (leading, financing or in reference group)
World Bank and EU has taken the lead in 85% of all assessments
Increasing frequency of joint work and partner inclusiveness
Governments have expressed appreciation of the objective nature of the scoring criteria of indicators. Government is able to do its own assessment or challenge assessments prepared by others.
Writing a fully evidenced assessment report is a challenge to governments in many countries where skilled professional staff is in very short supply. Where government leads, consultants are often drawn upon to write the report.

19. Coverage of PFM-PR in Reform Cycle
Implement PFM reforms
Recommend PFM reform measures
Identify main PFM weaknesses
High level performance overview
Investigate underlying causes
Formulate PFM reform program
PFM-PR
A PFM Reform program should be formulated from recommendations arising from an investigation having been undertaken to establish the weaknesses in the performance of PFM systems, processes and institutions. This high level performance overview of the PFM system can be obtained from the application of the PEFA framework. The diagram illustrate the coverage of the PFM report in the PFM reform cycle and the elements which it covers. The PFM report will embrace the high level performance overview and identify the main PFM weaknesses. This is a useful input to a reform dialogue with the aim of deciding reform priorities.
Detailed investigation of underlying causes for poor performance may be needed in the selected priority areas for reform. Specialized ‘drill-down’ assessment tools exist for many sub-systems and will help in providing inputs needed to determine a detailed action plan.
When reform action plans have been implemented for say 3 years it may be time to undertake a repeat assessment and measure change.

20. PEFA contribution to results orientation
Frequently used as basis for reform dialogue
Useful in deciding reform priorities
Ownership means government decisions on priorities
Indicators used to set targets for reform program results
but:
PEFA assessment is one of several inputs
Do not use Indicator scores simplistically
Complementary analysis to PEFA required to identify root causes
Measuring performance change
Results from 33 repeat assessments indicate pace of progress in different areas of PFM – positive global trend but wide variation among PFM features
Peer learning
Used as basis for discussions in regional groups of government PFM experts (e.g. Eastern Europe, West Africa, Caribbean)
Information from 25 countries (Impact study and External Evaluation 2010/11) indicate widespread use of PEFA assessments for reform dialogue by both governments and development agencies.
Government ownership of the assessment leads to government use of the report. Governments will typically need time digest the assessment findings, have an internal dialogue and formulate a position before they are ready to discuss with development partners.
Indicators are often used to set medium – long term targets for performance improvements.
However, other inputs are needed to formulate the reforms – political economy analysis, assessment of capacity factors, ongoing reforms.
Reform does not have to focus any or all of the low scoring PFM features. A ‘D’-scored feature may not necessarily be of major significance in a particular country environment. Guidance on an approach to sequencing PFM reforms is being developed.

21. Use of PEFA for Development Agencies’ Internal Processes
Used by all seven PEFA partners for internal processes (GBS/DPL, FRA, CPIA, Debt Limits etc)
Similarly used by several other development agencies
Increasingly, PEFA assessments are directly mentioned in internal guidelines as key or preferred input
PEFA assessment information is rarely considered the only source of information to a process
Most of the processes have financial consequences for partner countries – though the impact of PEFA is usually indirect.
other development agencies using PEFA assessments: regional banks (AfDB, IADB), and bilateral agencies e.g. Netherlands, Germany, Sweden, CIDA (Canada), AusAid etc.
internal guidelines : WB, EU, DFID, France. DFID specifically require that PEFA Secretariat has contributed a report review, before the report is used for further work.
PEFA assessments – the indicator ratings – provide essential inputs to many processes determine aid allocations by development agencies or choice of aid modality. In most cases, however, other inputs are also required so that the impact of PEFA is diluted. This is reassuring since the incentives for assessment manipulation would be too great.

22. Assessment comparisons
PEFA Framework developed to measure progress over time in one country
‘Summary assessment’ provides overview of strengths & weaknesses as basis for reform prioritization in a country
Comparison of assessments over time is robust where repeat assessments are explicitly comparing to baseline
Country comparisons
Comparing scores alone can be misleading
Must be approached cautiously - ensure comparison of like with like
Aggregation - No method given for arriving at single measure of ‘overall performance’ – no attempt to create ‘league table’
The monitoring report 2010 has proven that comparison of indicator rating over time is quite robust when the repeat assessment specifically was intended to compare to the baseline. It is easy to see where the indicator scores cannot be compared due to the baseline and/or repeat assessment assigning a ‘not used’, ‘not applicable’ or ‘not scored for lack of information’ to an indicator.
Country comparison requires caution. Best to compare to reports by reading them in parallel and compare indicator by indicator.
Many users wish to aggregate indicator ratings. Weighting of indicators and of countries pose problems. No method is perfect. PEFA program does not provide any method.
Country contexts are different, structure of public sector may be different, different definitions may apply, assessment time may differ etc.
The Program has wished to calculate country aggregate ratings and have a ’beauty contest’.

23. Harmonization of PFM diagnostic tools
Common information pool being created
Considerable use of PEFA assessments for many different purposes
Several earlier instruments replaced by PEFA
Transactions costs in PFM diagnostics appear not to decline, due to:
Still several overlapping assessment instruments
Many uncoordinated fiduciary assessments
Proliferation of new ‘drill-down’ PFM tools
Few countries have multi-year plans for analytical work
Three types of diagnostic instruments: broad PFM systems tools, speicalised drill-down tools, fiduciary risk assessment instruments
PEFA has replaced HIPC assessments, most CFAAs and EC compliance audits.
Broad based, overlapping assessment instruments : PEFA, OBI, Fiscal Transparency ROSC, Commonwealth PFM self asst tool kit etc.
new ‘drill-down’ PFM tools: tax administration (EC tax administration blueprints) , procurement (MAPS), debt management (DeMPA), etc.
Fiduciary instruments belong to individual development agencies.

24. THE PEFA Secretariat PEFA Secretariat Located in Washington DC.
Attached to the World Bank, which holds its funding in trust
Head of Secretariat reports directly to the PEFA Steering Committee
Staff of six full time

25. Role of the PEFA Secretariat
Neutral body
Supporting / advising any user of the Framework
does not represent a particular interest
does not undertake or finance assessments
support services are free of charge
The Secretariat’s clients include: Development agency staff, Partner government officials, private sector consultants, training institutes, researchers

26. Activities of the PEFA Secretariat (1)
Custodian of the PEFA Framework
Development and maintenance of the Framework
Issues guidance notes and clarifications
Quality reviews of assessments
Both at concept note and report stage, on request
Training
Develops and shares training materials
Supports training institutes in delivering PEFA courses
Delivers training selectively, mainly on a regional basis
Dissemination
Conference presentations, PFM blogs, PEFA Newsflash
Sharing PEFA assessment reports through website
The first change to Framework was issued January 2011 – three indicators revised
Clarification on the indicators are issued periodically.
Quality review is a core function.
Secretariat has limited capacity to deliver training. But often supporting others – e.g. co-facilitating
More than 100 assessment reports are linked to the PEFA website by hyperlinks. No direct posting because the secretariat does not own any of the reports.

27. Role of the PEFA Secretariat (2)
Monitoring
Semi-annual updates of PEFA assessment status list
Periodic monitoring reports
Ad hoc surveys
Promotes harmonization in assessment of PFM systems
Working with OECD-DAC Task Force on PFM
Stocktaking study of PFM diagnostic instruments
Supports developers of ‘drill-down’ assessment tools
Support PFM research
Availability of PEFA indicators database
Semi-annual status lists are provides a full overview of assessments by country according to stage (planned, ongoing, full draft report, final report, Public report) and stakeholders involved (lead agency and others).
Four major monitoring reports issue up to June 2011 (Early Experience Report, MR07, MR09, MR10)
Ad hoc surveys include use of PEFA for internal procedures and reports on publication of assessments
Secretariat provides advice to other developers of similar tools. Sharing experience. Encouraging clear linkages between tools.
Several research reports have been undertaken on the basis of PEFA scoring data. Secretariat capacity to undertake research is very limited. Also used for evaluation studies of general budget support and of technical assistance to PFM reform

28. PEFA Secretariat quality review
On request, free of charge, quick feedback (10 business days)
For Concept Notes/TOR and as well as Assessment Reports
Appraises adequacy of background info (sections 1, 2 & 4)
Appraises application of performance indicators (section 3)
Review of each indicator: correctly interpreted, sufficient evidence, scoring method correctly applied?
Considers whether summary assessment brings out clear message consistent with indicator analysis and background
Follow-up reviews – evaluates responses to the initial review
The Secretariat aims at providing comments on reports with ten business days of receiving request for review. For concept notes and terms of reference it is usually provided within a week.
When acting as Reviewers the PEFA Secretariat does not undertake an audit to establish the authenticity or correctness of the data given in the report. Reliance is placed on the integrity and professional competence of the consultants or team performing the assessment. However, where conflicting data or evidence becomes apparent then this will be pointed out and clarification sought.
The Secretariat’s review is undertaken by (i) appraising the application of the PFM performance indicators and (ii) considering whether the back-ground information as set out in chapters 1 & 2 of the framework has been given ; and whether the summary assessment brings out a clear message consistent with the indicator analysis.
Important to have other quality reviewers with specific country knowledge – so that country data may be verified.
Desirable to have reviewers from other agencies than the lead agency.
Secretariat undertakes follow-up reviews where the response to each of the earlier provided comments by the Secretariat is evaluated to determine if it is adequate.

29. Stay in touch with PEFA Visit the PEFA website: www.pefa.org
Send questions to the Secretariat:
Get on PEFA’s news distribution list: Send your name and address to the Secretariat