SAI Performance Measurement Framework Presented by: Luciano Danni Date:may, 2012 INTOSAI Working Group on the Value and Benefits of SAIs

Responsibilities and Reporting Lines INTOSAI Congress (2013) Approval INTOSAI Working Group on the Value and Benefits of SAIs Oversight and Direction Mandate: Johannesburg Accords (INCOSAI XX, 2010) INTOSAI Capacity Building Committee INTOSAI-Donor Steering Committee Other Relevant INTOSAI Committees In Consultation with: WGVBS SAI PMF Task Team Development of SAI Performance Measurement Framework Coordinator: INTOSAI-Donor Secretariat @ IDI SAI PMF Reference Group Technical Advice, Review and Quality Assurance INTOSAI Capacity Building Committee Members INTOSAI-Donor Steering Committee Members Other Groups: PEFA Secretariat, International Budget Partnership INTOSAI Capacity Building Committee Members INTOSAI-Donor Steering Committee Members

What is a Performance Measurement Framework? Tool for measuring performance …of an individual, system or organization …against an agreed performance scale …accommodating local circumstances To enable Better management of performance Monitoring over time Evaluation and lesson learning Contributing to Understanding performance Improving performance In this case, the ISSAIs and other INTOSAI good practice

What is the SAI Performance Measurement Framework? A Performance Measurement Framework tailored to SAIs Intended to give a high level assessment of SAI performance against established INTOSAI good practice: The ISSAI framework The Framework on the Value and Benefits of SAIs Other INTOSAI guidance material Consisting of a set of measurable indicators and guidance for a narrative performance analysis Combination of objective measurement and qualitative assessment

SAI PMF – How will it be used in Practice? Using the SAI PMF will be voluntary for the SAIs – the tool will be a public good which SAIs may make use of if it is helpful for them The SAIs themselves will make key decisions on how the assessment will be carried out The SAI will also decide whether or not to publish results Who decides to initiate a SAI PMF? Head of SAI Who decides when to do a SAI PMF? Head of SAI Consulting with INTOSAI regions & donors as appropriate Every 3-5 years recommended Head of SAI Approach selected to meet the SAI’s objectives 3 broad approaches Quality assurance to promote objectivity and credibility Who decides how to do a SAI PMF? Who decides if, when and how to publish a SAI PMF? Head of SAI

3 Approaches to Conducting a SAI PMF Assessment The SAI PMF will be equally well suited for three different ways of carrying out the assessment: 1. SAI Self Assessment Trained Team from within the SAI 2. INTOSAI Peer Review Peer SAI INTOSAI Regional Body 3. External Assessment Professional PFM Body Donor Agency Consultant External auditor . The approach will not necessarily affect how the detailed assessment is carried out… … but the SAI may use the results for different purposes depending on the approach

Why should the SAI use the PMF? As a step towards ISSAI implementation Increased understanding in the INTOSAI community of what good practice for SAIs entails Individual SAIs can benchmark their performance against good practice – and learn where there is room for improvement Based on this they can make more detailed ISSAIs compliance assessments To demonstrate progress and value and benefits to society The framework will be well suited for measuring progress over time and demonstrating this to external stakeholders Show to stakeholders how the SAI contributes to strengthening public financial management Internal performance measurement / annual reporting

Why should the SAI use the PMF? (2) To get support for capacity development efforts SAIs can use assessment results to get support from donors and other external stakeholders

SAI Performance Measurement Framework – Proposed Structure SAI Environment SAI Capability Impact SAI STRATEGY SAI Performance C. Strategy and Organizational Development B. Independence and Legal Framework Contribution to Public Financial Management, Governance and Tackling Corruption SAI CORE BUSINESS D. Audit Standards and Methodology E. Management & Support Structures F. Human Resources G. Communication and Stakeholder Management SAI SUPPORT SERVICES Public Financial Management Environment

A. SAI Performance SAI-1 Financial Audit Results SAI-2 Compliance Audit Results SAI-3 Performance Audit Results SAI-4 Other SAI Activities SAI-5 SAI Annual Report B. SAI Independence and Legal Framework SAI-6 Independence of the SAI SAI-7 Independence of the Head of the SAI and its Officials SAI-8 Mandate of the SAI C. Strategy and Organizational Development SAI-9 Strategy and Organizational Development

D. Audit Standards and Methodology SAI-10 Audit Plan for SAI SAI-11 Financial Audit Procedures SAI-12 Compliance Audit Procedures SAI-13 Performance Audit Procedures SAI-14 Quality Control and Quality Assurance of Audit Processes E. Management and Support Structures SAI-15 Management and Internal Control SAI-16 Support Structures and Infrastructure F. Human Resources SAI-17 Leadership and Management SAI-18 Planning and Policy Development SAI-19 Training and Knowledge Sharing

G. Communication and Stakeholder Management SAI-20 Communications Strategy and Internal Communication SAI-21 Communication with the government branches SAI-22 Communication with the Media, the Citizens and the Civil Society

The Audit Standards and Methodology-domain is the heart of the PMF The Audit Standards and Methodology-domain is the heart of the PMF. This covers financial, Compliance and performance audit. It measures the performance of the SAI against the level 3 of the ISSAIs.

Status of project The Working Group on the Value and Benefits of SAIs to discuss concept note and way forward at its meeting in Mexico 11-13 June 2012 Proposed way forward includes piloting in different regions during 2012 and 2013 First draft PMF including indicators completed, but needs further development

Coordination with PSC Current revision of ISSAIs may have implications for SAI PMF – should keep each other informed of developments PSC members welcome to join the Reference Group for the SAI PMF SAIs interested in conducting a pilot are invited to contact the INTOSAI Development Initiative

Further Information www.idi.no INTOSAI-Donor Cooperation SAI Performance Measurement Framework E-mail: INTOSAI.Donor.Secretariat@idi.no