QUALITY ASSURANCE AND IMPROVEMENT PROGRAM (QAIP) AGA Austin CPE Luncheon February 13, 2014 Presented by: Paul Morris, CIA, CPA Priscilla Suggs, MBA

Presentation Objectives Quality Assurance - what is it and why do we need a program? Understanding the Quality Assurance requirements per Red Book and Yellow Book standards Review an example of an Internal Audit QAIP process - DFPS

Audit Standards and GAGAS Standards (Red Book) 1300 – Quality Assurance and Improvement Program (QAIP) 1310 – Requirements of the QAIP 1311 – Internal Assessments 1312 – External Assessments 1320 – Reporting on the QAIP GAGAS (Yellow Book) Standards 3.82 through 3.107 - ‘Quality Control and Assurance’ Understand that QAIP is a Red Book term. Yellow book phraseology is ‘Quality Control and Assurance’ The requirements are similar…

Why do we need a QAIP? To ensure the Internal Audit Director (IAD) has established an internal audit activity “whose scope of work includes activities found in the Standards and in the Definition of Internal Auditing.” IPPF, Practice Advisory 1300-1

Why do we need a QAIP? “Each audit organization performing audits in accordance with GAGAS must: a. establish and maintain a system of quality control that is designed to provide the audit organization with reasonable assurance that the organization and its personnel comply with professional standards and applicable legal and regulatory requirements, and b. have an external peer review performed by reviewers independent of the audit organization being reviewed at least once every 3 years.” GAGAS, Standard 3.82

A QAIP can improve processes.

The QAIP Objective (Red Book) To provide reasonable assurance to our stakeholders that Internal Audit: Performs in accordance with the internal audit Charter Operates in an effective and efficient manner, and Is perceived by stakeholders as adding value and improving the organization’s operations IPPF, Practice Advisory 1300-1

The QAIP Objective (Red Book) It is an evaluation of the division’s processes It is comprehensive and covers all aspects of the operation and management of the internal audit activity, and It is performed by or under the direct supervision of the IAD IPPF, Practice Advisory 1300-1 The IAD is ultimately responsible for the QAIP, which covers all types of INTERNAL AUDIT activities, including consulting. In a small audit shop the IAD would most likely perform the assessments, but in an audit shop our size the QAIP responsibilities would be delegated to subordinates. In a large or complex environment, the IAD might establish a formal QAIP function, headed by an internal audit executive independent of the audit and consulting segments of the internal audit activity.

The QAIP Objective (Red Book) The QAIP is not an attempt to reinvent the wheel but rather to ensure that Internal Audit consistently provides quality and value-added services to its stakeholders. Begins with instituting policies, procedures and practices that are consistent with the Standards

Requirements of QAIP The quality assurance and improvement program must include both internal and external assessments IPPF, Standard 1310; Practice Advisory 1310-1 Ongoing and periodic assessment - Includes continuous supervision and periodically validating conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards. - Also… ongoing measurements and analyses of performance metrics (e.g., internal audit plan accomplishment, cycle time, recommendations accepted, and customer satisfaction). 2. Assessments evaluate and conclude on the quality of the internal audit activity and lead to recommendations for appropriate improvements. QAIPs include an evaluation of: Conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards, including timely corrective actions to remedy any significant instances of nonconformance. Adequacy of the internal audit activity’s charter, goals, objectives, policies, and procedures. Contribution to the organization’s governance, risk management, and control processes. Compliance with applicable laws, regulations, and government or industry standards. Effectiveness of continuous improvement activities and adoption of best practices. The extent to which the internal audit activity adds value and improves the organization’s operations. 3. The QAIP efforts also include follow-up on recommendations involving appropriate and timely modification of resources, technology, processes, and procedures. 4. Communicate the results of external and internal assessments to stakeholders. At least annually, the CAE reports to senior management and the board on the quality program efforts and results.

Internal Assessments Must include: Ongoing monitoring of the performance of the internal audit activity; and Periodic reviews performed through self-assessment or by other persons within the organization with sufficient knowledge of internal audit practices. IPPF, Standard 1311; Practice Advisory 1311-1 Interpretation: Ongoing monitoring – Part of the routine operation of an audit function. Is an integral part of the day-to-day supervision, review, and measurement of the internal audit activity. Ongoing monitoring is incorporated into the routine policies and practices used to manage the internal audit activity and uses processes, tools, and information considered necessary to evaluate conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards. Periodic reviews are assessments conducted to evaluate conformance with the Definition of Internal Auditing, the Code of Ethics, and the Standards. Sufficient knowledge of internal audit practices requires at least an understanding of all elements of the International Professional Practices Framework

Internal Assessments – Ongoing Monitoring Supervision of audits, regular, documented reviews of work papers; IA checklists and policies/procedures to ensure compliance with applicable standards; Feedback from customers (surveys) and other stakeholders; Selective workpaper peer reviews; Management tools: time budgets, time tracking systems, measuring audit plan completion Analysis of performance metrics. Practice Advisory 1311-1 Feedback from stakeholders. In addition to surveys, this could include the basic question when meeting with management: “ are we covering and documenting the risks that you feel are important?” Analysis of performance metrics: what should you look at to ensure/improve IA effectiveness and efficiency? We talked about measures during the strategic planning session. What are a few?

Internal Assessments – Periodic Reviews Stakeholder surveys and interviews; Can be performed by members of the IA activity; Can be performed by CIAs currently assigned elsewhere in the organization; Can include combination of self-assessment and preparation of materials for others to review; Benchmarking IA’s practice and performance against relevant best practices of the profession Practice Advisory 1311-1 Periodic Reviews: internal, and should have some validation performed sometime What do we do with the results? We develop actions to improve and conform with standards The CAE should put in place a reporting process and report annually to senior management/board.

Internal Assessments – GAGAS “Audit organizations should establish policies and procedures for monitoring of quality in the audit organization. Monitoring of quality is an ongoing, periodic assessment of work completed on audits designed to provide management of the audit organization with reasonable assurance that the policies and procedures related to the system of quality control are suitably designed and operating effectively in practice.” GAGAS, Standard 3.93 3.94: “Monitoring procedures will vary based on the audit organization’s facts and circumstances.” and 3.95: “The audit organization should analyze and summarize the results of its monitoring process at least annually, [with noted issues and recommendations].

External Assessments Must be conducted at least once every five years by a qualified, independent reviewer or review team from outside the organization. The reviewer or review team should be qualified, independent and from outside the agency. IPPF, Standard 1312; Practice Advisory 1312-1 “conducted at least once every five years…” Can they be done more often? Sure, if needed, and sometimes they are.  Important: Independence – no real or apparent conflict of interest and not being part of or controlled by the organization to which the reviewed audit activity belongs

External Assessments “The audit organization should obtain an external peer review at least once every 3 years that is sufficient in scope to provide a reasonable basis for determining whether, for the period under review, the reviewed audit organization’s system of quality control was suitably designed and whether the audit organization is complying with its quality control system in order to provide the audit organization with reasonable assurance of conforming with applicable professional standards.” GAGAS, Standard 3.96

Considerations for External Review The qualifications of external reviewers as noted in The IIA’s Practice Advisory 1312-1 should be considered when contracting with an outside party to conduct the assessment. The qualified reviewer or review team has to demonstrate competence in two areas: the professional practice of internal auditing and the external assessment process. In the case of a review team, not all members of the team need to have all the competencies; it is the team as a whole that is qualified. Show PA 1312-1 which is very extensive – signifies the importance of selecting an independent external reviewer. GAGAS 3.104 also addresses the criteria for the Peer Review Team

Scope of the External Assessment Conformance with the Standards, Definition of Internal Auditing, the Code of Ethics, and internal audit’s Charter, plans policies, procedures, practices, and any applicable legislative and regulatory requirements. Expectations of Internal Audit as expressed by the Governance and Management (Executive Team) Integration of the Internal Audit activity into DFPS’s governance process, including the audit relationship between and among the key groups involved in the process.

Scope of the External Assessment Tools and techniques used by Internal Audit. The mix of knowledge, experiences, and disciplines within the staff, including staff focus on process improvement. A determination whether Internal Audit adds value and improves DFPS’s operations. IPPF Practice Advisory 1312-1

Reporting on the QAIP Internal Assessment Reporting Results of internal assessments will be reported to the Audit Committee and to senior management at least annually. IPPF, Standard 1320 and Interpretation External Assessment Reporting Results which include the reviewer’s or review team’s assessment of conformance, is communicated to senior management upon completion. Internal Assessment Reporting: Red book and Yellow book are in sync requiring annual reporting: Standards 1311 and GAGAS 3.95 IPPF, Standard 1320: “The chief audit executive must communicate the results of the quality assurance and improvement program to senior management and the board.”

Implementing Corrective Action If there are any recommendations, the IAD should implement appropriate follow-up actions to ensure action plans are developed and implemented in a reasonable timeframe.

Example – Internal Monitoring Process DFPs Internal Audit uses a team approach to the annual internal monitoring approach. Each team member is assigned a portion of the QAIP, conducts review, and reports results to the Internal Audit Director. The Director consolidates results and reports annually to the Commissioner and DFPS Executive Leadership Team. Copies of the DFPS QAIP policy/procedure, assignments and final report are included for discussion. Internal Assessment Reporting: Red book and Yellow book are in sync requiring annual reporting: Standards 1311 and GAGAS 3.95 External Reporting:

Questions?