1. Support for strengthening Quality Assurance
Karma Tenzin
Manager Capacity Development, IDI

2. Will cover….. Why is QA important ? Setting up a SAI level QA function
Conducting a QAR
IDI support for strengthening QA

3. Quality Control Organisation level (ISSAI 40)
The organisation is required to establish & maintain a system of quality control to ensure the organisation and its personnel complies with professional standards and applicable legal and regulatory requirements; and issues reports that are appropriate in the circumstances.
Includes elements of quality control to be; leadership, ethics, acceptance and continuance, human resources, engagement performance, and monitoring.
Individual engagement level (respective ISSAIs)
Quality Control
ISSAI 40 is based on the key principles in the ISQC1 adapted as necessary to apply to SAIs. ISSAI 40 reflects the mandate of SAIs which is often wider than that of professional audit and assurance firms. ISSAI 40 outlines quality control measures that are relevant to achieving high quality in public sector environment.
ISSAI 40 focuses on organisational aspects of audit quality throughout SAIs. ISSAI 40 also provides a framework that complements quality control at an audit engagement level (eg: an individual financial audit, compliance audit, performance audit or any other work carried out by an SAI).
As an overriding objective, each SAI should consider the risks to the quality of its work and establish a system of quality control that is designed to adequately responded to these risks. These risks may arise in many different aspects of an SAI’s work. For example risks to quality may arise in the application of professional judgement, the design and implementation of policies and procedures, or in the methods used by SAIs to communicate the results of their work.
Quality control standards at an individual audit engagement level are:
Financial Audit – ISSAI 1220 – Quality Control for an audit of financial statements (Specific standard) – refers application and other explanatory material – System of Quality Control and Role of Engagement Teams (Assignment of engagement teams - refer Para A10 to A12, and Engagement Performance – refer A13 to A22 – direction, supervision, performance, and reviews – established clear lines of responsibilities within the engagement team – this is QC process in an audit engagement team – the audit supervisor or an engagement partner has the overall responsibility-reviews performance have been documented. We support by suggesting this process in our FA ISSAI Implementation Handbook.
Compliance Auditing – under General Requirements of compliance auditing ISSAI 4000 (Requirement No. 80 – quality control process and 85 – audit team competence).
Performance audit – Under General Requirements of Performance Auditing ISSAI 3000-Para 79 and further guidance provided in ISSAI Para – related to quality control of performance audit process.

4. Quality Assurance ISSAI 40.6 - Quality assurance after the fact
Monitoring process
Designed to provide the SAI with reasonable assurance
That the policies and procedures relating to the quality control system are relevant, adequate, and operating effectively in practice
Audit was conducted in accordance with the professional standards.
Maintaining a system of quality control requires ongoing monitoring and a commitment to continuous improvement – the 6th element of the ISSAI 40, i.e. monitoring, in other words, the QA review process.
The monitoring process should as per ISSAI 40:
Include an ongoing consideration and evaluation of the SAI’s system of quality control, including a review of a sample of completed work across the range of work carried out by the SAI;
require responsibility for the monitoring process to be assigned to an individual or individuals with sufficient and appropriate experience and authority in the SAI to assume that responsibility; and
Require that those carrying out the review are independent (i.e. they have not taken part in the work or any quality control review of work)
Independent review could be done by either internal team independent of audit team or by external reviewer.
The subject of the QA review is the SAIs Quality Control System at both SAI level and an individual audit engagement level.
IFAC has QA standards for its members under the Statement of Membership Organisation (SMO 1: Quality Assurance) – As per this standard, the subject of the QA review could be either; 1) a Firm or 2) a Partner.

5. SAI PMF data shows that 21% of SAIs met the benchmarks for quality assurance policies, 18% satisfactorily implemented these in practice.
QA demonstrates that QC policies and procedures are put into place and practiced, and therefore gives evidence of the extent of ISSAI compliance.
The information is collected and analysed from the SAI PMF Assessments while reviewing the implementation of current level 2 ISSAIs: ie. Independence, Ethics, Accountability and Transparency, Quality Control and Quality Assurance and reflected in the Global SAI Stocktaking Report 2017.
In August 2017, IDI conducted an analysis of 25 SAI PMF assessments of developing countries to examine ISSAI Implementation (refers Page 34 of SAI Stocktaking report 2017).
Refer page 32 of the report – 71% of the SAIs reported that their QA system covers all three audit streams.
The quality control system is the condition for performing QA – an independent mechanism. QA is addressed at two levels; SAI level and the audit engagement level
INTOSAI establishes standards on quality control policies and procedures for:
SAI level-ISSAI 40 Quality Control for SAI
Audit engagements – ISSAI 1220 for FA and, relevant sections in ISSAI 3000 and ISSAI 4000
Why is QA important ?

6. Quality Assurance and Methodology
Implementation of recommendations will impact on methodology
Recommendations will be made including:
Additional quality control reviews
Training of staff
Changes to the SAI’s Quality Control policies and procedures
Disciplinary action
A Quality Assurance process provides a report that will flag deficiencies that require corrective action
This is quite related to the theme of this meeting – how the QA and Audit Methodology are related.
Quality Assurance and Methodology

7. Setting up a SAI level QA function
Assessing the need for a QA Function in the SAI 1
Developing and maintaining a QA Policy 2
Creating staff awareness of the QA Function 3
Setting up the QA Function 4
Managing the QA Function 5
Assessing the need for QA – The IDI in collaboration with the selected SAIs will carry out an assessment of the need for QA system at the SAI considering the environment in which it operates and the availability of resources to sustain the QA function. This exercise may also provide a basis to initiate appropriate interventions to create an enabling environment for setting up QA function. SAI PMF report if any can be referred in carrying out this assessment.
Developing QA policy - The SAI will develop a QA policy, which will provide the basis and overall framework of conducting the QA review on a regular basis. The policy will also become the main basis for developing the QA manual of the SAI, and determine the structure of QA function. The IDI will provide a guidance to develop the QA policy, and will facilitate online to develop such a policy.
Creating staff awareness of the QA function - The teams selected for different audit types (based on current audit practices in SAI) will be trained on overall setting up of the QA function and conducting the QA at audit engagement level. The training will cover both audit methodology for three types of audit and conducting QA. The training will be delivered using IDI’s QA guidance and tool, and course material developed for this purpose. The IDI will train the QA team on soft skills aspects as well. The QA team trained will support the SAI to establish the QA function, and will be a team that will advocate the need and importance of QA function at the SAI through training and other awareness programme. The IDI will facilitate this training at the selected SAI’s premises, and will be assisted by the SAI mentor team.
Setting up of the QA function - The QA policy developed by the SAI will define realistic structure of QA function. The SAI will set up the QA function as envisioned in the policy, and based on the experience from the pilot QA. However, it may take time to establish a fully functional QA department or division with adequate staff and other resources. The SAI will follow a gradual process of setting up a fully functional QA department or division. The SAI will identify some critical change needed to set up the QA function including restructuring of organisation, repositioning of staff, audit practices and management, etc. The SAI will then propose strategies to manage the change.
Managing QA Function – Ideally an independent function to exist, but could have different option of either internal function or having external reviewer.

8. Experience in SAIs on setting up QA function?

9. The Quality Assurance Review Process at the audit engagement level
Planning
Determine objective
Prepare overall strategy
Scope, timeline and approach
Understanding of audit engagement
Understanding of audit methodology
QA review methodology 
Design review procedures
Conducting
Apply QA tools
Perform General Procedures
Gather Evidence
Monitor quality of review
Completion and Review
Evaluate findings
Draw conclusions and propose recommendations
Discuss with Audit Engagement Team
Reporting
Discuss Preliminary Findings with Management
Prepare draft report
Conduct exit meeting
Obtain comments from Management
Incorporate comments of management in the report
Finalize & issue report
Follow-up
Review the implementation of the recommendations
Issue Follow-up report
Refer QA guidance to explain these steps in brief
These are the suggested steps in our guidance

10. Experience in SAIs on QA review at audit engagement level?

11. IDI Support for Quality Assurance
Component of Professional SAIs workstream
Component of 3i Programme
The QA programme is one of the elements of the 3i Programme – the ISSAI Implementation Initiative
In the new Strategic Plan – it is one of the components under Professional SAIs Workstream (support for SAI level, Cooperative Audit and developing Global Public Goods) – this will be a continuous basis of support to SAIs under the workstream approach.
IDI Support for Quality Assurance

12. IDI support for Quality Assurance
IDI Vision, Mission and Principles
IDI support for Quality Assurance
Global Public Goods on Quality Assurance
QA Tools and Guidance
QA tools for FA, PA and CA based on ISSAIs
Common guidance on conducting QA

13. Global Public Goods on Quality Assurance
QA tools for FA, PA and CA based on ISSAIs
Identification of applicable ISSAIs
Questions covering applicable ISSAIs
Since ISSAIs for financial audits are applied in both private and public sectors, we have introduced a section in the FA QA tool-the mapping of applicable ISSAIs to audit engagements selected for the QA review. This is expected to reduce the burden of QA reviewer to go through every 37 standards and requirements. This will then identify the applicable ISSAIs to a particular audit engagement.
Suggested list of questions (the checklist) against each essence of the requirements.
Global Public Goods on Quality Assurance

14. Financial Audit QA Tool
Demonstrate the functionality of FA QA tool
Tool developed writing Macros to automate the functionality wherever possible and to make it more efficient for the QA reviewer

15. Performance Audit QA Tool
- Show PA QA tool

16. Compliance Audit QA Tool
- Show CA QA tool

17. IDI support for Quality Assurance
IDI Vision, Mission and Principles
IDI support for Quality Assurance
Professional Education for SAI Auditors ( PESA) Pilot
Syllabus for cross cutting competencies includes education on quality control and quality assurance
Emphasis on ‘Quality’ as cross cutting aspect in the syllabus for all three functional competencies (Financial Audit, Performance Audit, Compliance Audit)
PESA is one of the elements under Professional SAI workstream
Education programme to implement ISSAI in SAIs – building the professional capacity of SAI auditors
QC and QA elements are central to the syllabus of the PESA

18. IDI support for Quality Assurance
IDI Vision, Mission and Principles
IDI support for Quality Assurance
QA in Cooperative Audit Support
QA Review of audit included in SAI commitments for Cooperative audits
Arrangements made to conduct QA reviews by independent, competent QA reviewers.
QA review of Cooperative Audits of Disaster Management in ASOSAI and OLACEFS in 2017 (the Performance Audits)
QA review of Cooperative Financial Audit in ARABOSAI in 2017
QA is the part of the Cooperative Audit Model
We arrange independent and having adequate competence to conduct the QA review of these cooperative audits using IDI guidance and the QA tool.
Besides cooperative audits, we have also supported the QA review of SAI level support in SAI Bhutan in 2017 for six pilot audits (2 each for FA, CA and PA)

19. IDI support for Quality Assurance
IDI Vision, Mission and Principles
IDI support for Quality Assurance
3 SAIs to be selected based on readiness and commitment
Assessment of QA needs
Support will be tailored to needs . Could include support in developing QA policy, develop QA manual, develop QAR plans, train QA reviewers and pilot QAs
SAI Level support for setting up QA function
Support for setting up QA is the final component of SAI Level ISSAI Implementation support model
Such support will be based on assessed needs of the SAI
SAI Level ISSAI Implementation Support
SAI-level QA support
We have been providing support for setting up QA function as a part of SAI level support in SAI Bhutan and SAI Tonga till now
Going forward, we plan to provided support following two approaches, i.e. the standalone support for setting up QA function and as a part of holistic support for SAI level support ISSAI Implementation where setting up of QA function will come at the end of the process.

20. IDI support for Quality Assurance
IDI Vision, Mission and Principles
IDI support for Quality Assurance
50 QA Reviewers trained for FA, PA, CA in English Speaking INTOSAI regions
Regional Pool of QA Reviewers
( PASAI 2018)
QA Reviews as a part of IDI- Region Cooperative Audits
Support to the regions

21. Questions