Support for strengthening Quality Assurance Karma Tenzin Manager Capacity Development, IDI

Will cover….. Why is QA important ? Setting up a SAI level QA function Conducting a QAR IDI support for strengthening QA

Quality Control Organisation level (ISSAI 40) The organisation is required to establish & maintain a system of quality control to ensure the organisation and its personnel complies with professional standards and applicable legal and regulatory requirements; and issues reports that are appropriate in the circumstances. Includes elements of quality control to be; leadership, ethics, acceptance and continuance, human resources, engagement performance, and monitoring. Individual engagement level (respective ISSAIs) Quality Control ISSAI 40 is based on the key principles in the ISQC1 adapted as necessary to apply to SAIs. ISSAI 40 reflects the mandate of SAIs which is often wider than that of professional audit and assurance firms. ISSAI 40 outlines quality control measures that are relevant to achieving high quality in public sector environment. ISSAI 40 focuses on organisational aspects of audit quality throughout SAIs. ISSAI 40 also provides a framework that complements quality control at an audit engagement level (eg: an individual financial audit, compliance audit, performance audit or any other work carried out by an SAI). As an overriding objective, each SAI should consider the risks to the quality of its work and establish a system of quality control that is designed to adequately responded to these risks. These risks may arise in many different aspects of an SAI’s work. For example risks to quality may arise in the application of professional judgement, the design and implementation of policies and procedures, or in the methods used by SAIs to communicate the results of their work. Quality control standards at an individual audit engagement level are: Financial Audit – ISSAI 1220 – Quality Control for an audit of financial statements (Specific standard) – refers application and other explanatory material – System of Quality Control and Role of Engagement Teams (Assignment of engagement teams - refer Para A10 to A12, and Engagement Performance – refer A13 to A22 – direction, supervision, performance, and reviews – established clear lines of responsibilities within the engagement team – this is QC process in an audit engagement team – the audit supervisor or an engagement partner has the overall responsibility-reviews performance have been documented. We support by suggesting this process in our FA ISSAI Implementation Handbook. Compliance Auditing – under General Requirements of compliance auditing ISSAI 4000 (Requirement No. 80 – quality control process and 85 – audit team competence). Performance audit – Under General Requirements of Performance Auditing ISSAI 3000-Para 79 and further guidance provided in ISSAI 3100- Para 100-105 – related to quality control of performance audit process.

Quality Assurance ISSAI 40.6 - Quality assurance after the fact Monitoring process Designed to provide the SAI with reasonable assurance That the policies and procedures relating to the quality control system are relevant, adequate, and operating effectively in practice Audit was conducted in accordance with the professional standards. Maintaining a system of quality control requires ongoing monitoring and a commitment to continuous improvement – the 6th element of the ISSAI 40, i.e. monitoring, in other words, the QA review process. The monitoring process should as per ISSAI 40: Include an ongoing consideration and evaluation of the SAI’s system of quality control, including a review of a sample of completed work across the range of work carried out by the SAI; require responsibility for the monitoring process to be assigned to an individual or individuals with sufficient and appropriate experience and authority in the SAI to assume that responsibility; and Require that those carrying out the review are independent (i.e. they have not taken part in the work or any quality control review of work) Independent review could be done by either internal team independent of audit team or by external reviewer. The subject of the QA review is the SAIs Quality Control System at both SAI level and an individual audit engagement level. IFAC has QA standards for its members under the Statement of Membership Organisation (SMO 1: Quality Assurance) – As per this standard, the subject of the QA review could be either; 1) a Firm or 2) a Partner.

SAI PMF data shows that 21% of SAIs met the benchmarks for quality assurance policies, 18% satisfactorily implemented these in practice. QA demonstrates that QC policies and procedures are put into place and practiced, and therefore gives evidence of the extent of ISSAI compliance. The information is collected and analysed from the SAI PMF Assessments while reviewing the implementation of current level 2 ISSAIs: ie. Independence, Ethics, Accountability and Transparency, Quality Control and Quality Assurance and reflected in the Global SAI Stocktaking Report 2017. In August 2017, IDI conducted an analysis of 25 SAI PMF assessments of developing countries to examine ISSAI Implementation (refers Page 34 of SAI Stocktaking report 2017). Refer page 32 of the report – 71% of the SAIs reported that their QA system covers all three audit streams. The quality control system is the condition for performing QA – an independent mechanism. QA is addressed at two levels; SAI level and the audit engagement level INTOSAI establishes standards on quality control policies and procedures for: SAI level-ISSAI 40 Quality Control for SAI Audit engagements – ISSAI 1220 for FA and, relevant sections in ISSAI 3000 and ISSAI 4000 Why is QA important ?

Quality Assurance and Methodology Implementation of recommendations will impact on methodology Recommendations will be made including: Additional quality control reviews Training of staff Changes to the SAI’s Quality Control policies and procedures Disciplinary action A Quality Assurance process provides a report that will flag deficiencies that require corrective action This is quite related to the theme of this meeting – how the QA and Audit Methodology are related. Quality Assurance and Methodology

Setting up a SAI level QA function Assessing the need for a QA Function in the SAI 1 Developing and maintaining a QA Policy 2 Creating staff awareness of the QA Function 3 Setting up the QA Function 4 Managing the QA Function 5 Assessing the need for QA – The IDI in collaboration with the selected SAIs will carry out an assessment of the need for QA system at the SAI considering the environment in which it operates and the availability of resources to sustain the QA function. This exercise may also provide a basis to initiate appropriate interventions to create an enabling environment for setting up QA function. SAI PMF report if any can be referred in carrying out this assessment. Developing QA policy - The SAI will develop a QA policy, which will provide the basis and overall framework of conducting the QA review on a regular basis. The policy will also become the main basis for developing the QA manual of the SAI, and determine the structure of QA function. The IDI will provide a guidance to develop the QA policy, and will facilitate online to develop such a policy. Creating staff awareness of the QA function - The teams selected for different audit types (based on current audit practices in SAI) will be trained on overall setting up of the QA function and conducting the QA at audit engagement level. The training will cover both audit methodology for three types of audit and conducting QA. The training will be delivered using IDI’s QA guidance and tool, and course material developed for this purpose. The IDI will train the QA team on soft skills aspects as well. The QA team trained will support the SAI to establish the QA function, and will be a team that will advocate the need and importance of QA function at the SAI through training and other awareness programme. The IDI will facilitate this training at the selected SAI’s premises, and will be assisted by the SAI mentor team. Setting up of the QA function - The QA policy developed by the SAI will define realistic structure of QA function. The SAI will set up the QA function as envisioned in the policy, and based on the experience from the pilot QA. However, it may take time to establish a fully functional QA department or division with adequate staff and other resources. The SAI will follow a gradual process of setting up a fully functional QA department or division. The SAI will identify some critical change needed to set up the QA function including restructuring of organisation, repositioning of staff, audit practices and management, etc. The SAI will then propose strategies to manage the change. Managing QA Function – Ideally an independent function to exist, but could have different option of either internal function or having external reviewer.

Experience in SAIs on setting up QA function?

The Quality Assurance Review Process at the audit engagement level Planning Determine objective Prepare overall strategy Scope, timeline and approach Understanding of audit engagement Understanding of audit methodology QA review methodology  Design review procedures Conducting Apply QA tools Perform General Procedures Gather Evidence Monitor quality of review Completion and Review Evaluate findings Draw conclusions and propose recommendations Discuss with Audit Engagement Team Reporting Discuss Preliminary Findings with Management Prepare draft report Conduct exit meeting Obtain comments from Management Incorporate comments of management in the report Finalize & issue report Follow-up Review the implementation of the recommendations Issue Follow-up report Refer QA guidance to explain these steps in brief These are the suggested steps in our guidance

Experience in SAIs on QA review at audit engagement level?

IDI Support for Quality Assurance Component of Professional SAIs workstream Component of 3i Programme The QA programme is one of the elements of the 3i Programme – the ISSAI Implementation Initiative In the new Strategic Plan – it is one of the components under Professional SAIs Workstream (support for SAI level, Cooperative Audit and developing Global Public Goods) – this will be a continuous basis of support to SAIs under the workstream approach. IDI Support for Quality Assurance

IDI support for Quality Assurance IDI Vision, Mission and Principles IDI support for Quality Assurance Global Public Goods on Quality Assurance QA Tools and Guidance QA tools for FA, PA and CA based on ISSAIs Common guidance on conducting QA

Global Public Goods on Quality Assurance QA tools for FA, PA and CA based on ISSAIs Identification of applicable ISSAIs Questions covering applicable ISSAIs Since ISSAIs for financial audits are applied in both private and public sectors, we have introduced a section in the FA QA tool-the mapping of applicable ISSAIs to audit engagements selected for the QA review. This is expected to reduce the burden of QA reviewer to go through every 37 standards and requirements. This will then identify the applicable ISSAIs to a particular audit engagement. Suggested list of questions (the checklist) against each essence of the requirements. Global Public Goods on Quality Assurance

Financial Audit QA Tool Demonstrate the functionality of FA QA tool Tool developed writing Macros to automate the functionality wherever possible and to make it more efficient for the QA reviewer

Performance Audit QA Tool - Show PA QA tool

Compliance Audit QA Tool - Show CA QA tool

IDI support for Quality Assurance IDI Vision, Mission and Principles IDI support for Quality Assurance Professional Education for SAI Auditors ( PESA) Pilot Syllabus for cross cutting competencies includes education on quality control and quality assurance Emphasis on ‘Quality’ as cross cutting aspect in the syllabus for all three functional competencies (Financial Audit, Performance Audit, Compliance Audit) PESA is one of the elements under Professional SAI workstream Education programme to implement ISSAI in SAIs – building the professional capacity of SAI auditors QC and QA elements are central to the syllabus of the PESA

IDI support for Quality Assurance IDI Vision, Mission and Principles IDI support for Quality Assurance QA in Cooperative Audit Support QA Review of audit included in SAI commitments for Cooperative audits Arrangements made to conduct QA reviews by independent, competent QA reviewers. QA review of Cooperative Audits of Disaster Management in ASOSAI and OLACEFS in 2017 (the Performance Audits) QA review of Cooperative Financial Audit in ARABOSAI in 2017 QA is the part of the Cooperative Audit Model We arrange independent and having adequate competence to conduct the QA review of these cooperative audits using IDI guidance and the QA tool. Besides cooperative audits, we have also supported the QA review of SAI level support in SAI Bhutan in 2017 for six pilot audits (2 each for FA, CA and PA)

IDI support for Quality Assurance IDI Vision, Mission and Principles IDI support for Quality Assurance 3 SAIs to be selected based on readiness and commitment Assessment of QA needs Support will be tailored to needs . Could include support in developing QA policy, develop QA manual, develop QAR plans, train QA reviewers and pilot QAs SAI Level support for setting up QA function Support for setting up QA is the final component of SAI Level ISSAI Implementation support model Such support will be based on assessed needs of the SAI SAI Level ISSAI Implementation Support SAI-level QA support We have been providing support for setting up QA function as a part of SAI level support in SAI Bhutan and SAI Tonga till now Going forward, we plan to provided support following two approaches, i.e. the standalone support for setting up QA function and as a part of holistic support for SAI level support ISSAI Implementation where setting up of QA function will come at the end of the process.

IDI support for Quality Assurance IDI Vision, Mission and Principles IDI support for Quality Assurance 50 QA Reviewers trained for FA, PA, CA in English Speaking INTOSAI regions Regional Pool of QA Reviewers ( PASAI 2018) QA Reviews as a part of IDI- Region Cooperative Audits Support to the regions

Questions