1. SAI Performance Measurement Framework
Event
Location, Date
Presented By:

2. What is SAI PMF ?
SAI PMF is a tool for measuring, monitoring, managing and reporting the performance of a Supreme Audit Institution.

3. Purpose of a SAI PMF Assessment
Why might SAIs want to measure their performance
Identify strengths and weaknesses
Introduce / strengthen internal performance management and accountability
Enable external (annual) reporting to stakeholders
Demonstrate progress in capability and performance
Demonstrate the value and benefits of the SAI to citizens
Support ISSAI implementation / improve audit standards & quality
Obtain & maintain external support for capacity development
MULTIPLE PURPOSES FOR SAI PMF ASSESSMENTS
Note here that the SAI PMF has been designed as a GENERAL PURPOSE FRAMEWORK. It can be used for any of the above purposes. But the intended purpose of an assessment in a specific country has a profound implication on how the assessment should be best carried out to meet that purpose.

4. SAI PMF: Evidence Based Assessment of Performance
Holistic assessment of the SAI and its environment – including factors which are not covered by the indicators
Qualitative assessment (Performance Report)
Measurable indicators
Objective measurement to inform qualitative assessment and track progress over time
PEFA style drill down into SAI Performance

5. Why Develop a Global Framework for Measuring SAI Performance?

6. ISSAI Framework adopted in 2010
NEW Level 3:
Updated audit principles, consistent with level 4 1 2 3 4
Founding Principles
Prerequisites for the functioning of SAIs
Level 4:
First agreed, international public sector external audit guidelines
Fundamental
auditing Principles
For the first time, the ISSAIs provided a set of agreed international standards which most countries aspire towards, and against which performance could be measured. Prior to the adoption of the ISSAI framework, it was not possible to get agreement on what constitutes good performance, making development and agreement of a measurement tool impossible.
Auditing
guidelines

7. Why SAI PMF?
ISSAI 20 calls for SAIs to assess and report publically on their performance
No internationally agreed common framework for measuring SAI’s performance
The INTOSAI Congress (2010), decided to develop a framework for measuring SAI performance (PMF)
Key Points: SAIs globally are facing the challenge of how to assess their own performance, for both internal and external purposes. Includes the challenge of measuring performance changes over time, as well as how to best learn from the experiences of peers.

8. Progress made in the development of SAI PMF?
Version 2.0 published 2012 and piloted in Djibouti, Sierra Leone & Norway
Global consultation on version 2.0 in 2012
Updated SAI PMF Pilot Version published July 2013
11 SAI PMF training courses delivered in Europe, Africa, North & South America, Asia, Middle East – in English, French, Spanish and Arabic
Coming soon – Portuguese and Russian, the Pacific

9. Phase 2 Pilots - Volunteers
Completed:
Planned:
Slovakia
Argentina
Armenia
Guatemala
Bangladesh
Brazil
Burkina Faso
Costa Rica
In Progress:
Dominican Republic
Barbados
Haiti
Bhutan
Ecuador
Latvia
India
Maldives
Mexico
Mozambique
Expressed Interest:
Oman
SAIs across the Pacific
Nepal
Palestine

10. Adoption by INTOSAI Congress in 2016
What is next?
20 more pilot tests
More SAI PMF training courses
Further improve SAI PMF
Adoption by INTOSAI Congress in 2016

11. Part 1 - SAI PMF Indicator Framework

12. SAI Performance Measurement Framework Structure
7 Domains
SAI Capability
C. Strategy for Organizational Development
SAI Reporting
Value and Benefits of Supreme Audit Institutions
B. Independence and Legal Framework
D. Audit Standards and Methodology
E. Management & Support Structures
F. Human Resources and Leadership
G. Communication and Stakeholder Management
Note for facilitator – everything within the dotted black lines is under the control of the SAI (providing it is adequately resourced). If it is not within the control of the SAI due to specific independence issues, then we do not score it here and the limitations on SAI independence is reflected in the scores under domain B. Thus the part within the dotted lines follows the principle of only measuring things which the SAI is responsible for.
Everything within the red line is measured. Things outside the red line form part of the narrative assessment, because they are important, but are not directly measured – being difficult to measure against a standardised framework.
Country Context, Governance and Public Financial Management

13. Domain
Indicator 1
Dimension 1
Criteria 1
Criteria 2
Criteria 3
Dimension 2
Dimension 3
Dimension 4
Indicator 2
Criteria are taken directly from the ISSAIs or other international good practices

14. Part 2 - SAI PMF Assessment Report

15. From Evidence to Executive Summary
II Quality Assurance
IV Management Use of Assessment Results
Judgment
I Executive Summary
Analysis
III. Observations on SAI Performance & Impact
Within (III) there are four parts, as follows:
a) Is about understanding linkages between domains & what constrains SAI performance
b) Goes a step further than (a), to identify how the SAI’s performance makes a difference to the lives of citizens – specifically looking at linkages between SAI PMF indicators and the V&B framework, as well as identifying specific examples from the SAIs own reports (annual report, performance audit reports etc.)
c) Examines how factors largely inside or outside the SAI’s control support or undermine the SAIs ability to deliver impact, including constraints in its mandate and independence, weaknesses in the PFM and governance systems, country limitations, and the availability and effectiveness of external support
d) Looks at capacity development prospects, based on ch.4 and challenges identified in ch.3
Evidence
Ch.1 Introduction
Ch.2 Country & SAI Background
Ch.3 Assessment (Indicator Scores & Evidence)
Ch.4 SAI Capacity Development
Main Report

16. Part 3 – Initiating a SAI PMF Pilot Assessment

17. Six Key Decisions to be Made by the SAI
Undertake SAI PMF?
Assessment owner?
Purpose(s) of the Assessment?
Assessment Approach?
5. Timing?
6. Use of Results? 1 2 3 4 5 6
There are more decisions during the course of the assessment, but these are the six key ones to be taken at the outset, and made specifically by the SAI.

18. Learning More SAI PMF Training Course Part A – for assessors, 2.5 days
Part B – deeper understanding and future trainers, 2 days
Request a course:
The training course covers many issues not dealt with here, e.g.
Determining the scope of assessments, including dealing with regional offices and sub-national audit entities
Steps in the assessment process
Planning assessments and designing terms of reference
Quality assurance arrangements
Writing the SAI performance report