The Canadian USOAP Experience

The Canadian USOAP Experience Aim: to share Canadian experience gained from the ICAO audit process to provide other states with an insight that may be helpful in preparing for upcoming audits Outline Overview of the ICAO audit process Canadian preparation Canada’s Audit experience Lessons Learned Pitfalls to avoid Link: Canada was the first state to be audited by ICAO’s Universal Safety Oversight Audit Program in the new expanded form addressing 16 of the 18 Annexes and this presentation is being provided to describe our experiences with that process. The Aim is to share our experiences gained through this process to provide other states with an insight that may be helpful in preparing for your upcoming audits. To accomplish this, the format of this presentation will include a description of: - an overview on the ICAO audit process; - Canada’s preparation for the audit; - the actual audit experience; - and a summary of the lessons learned and pitfalls to avoid RDIMS/SGDDI

Overview of the ICAO audit process Program Objective (Assembly Resolution A35-6) Assess State’s implementation of: - Safety oversight, and - Standards and Recommended Practices, associated procedures and guidance material ICAO’s Universal Safety Oversight program evolved from the voluntary role beginning in 1995 addressing the 3 annexes, 1,6 and 8, to their current mandatory program addressing the 16 Safety related Annexes. Their program objective is to promote global aviation safety by auditing contracting states to determine the status of a State’s implementation of: Safety Oversight obligations and; the relevant Standards and Recommended Practices, associated procedures and guidance material. The words Recommended Practices and Guidance Material on this slide are underlined to draw attention to the fact the Assembly Resolution has mandated their program to audit beyond the Standards requirement to include material that some states may not view as an ICAO commitment. RDIMS/SGDDI

ICAO auditing Philosophy cont’d ICAO’s Comprehensive Systems approach: Phase 1. Review of State Aviation Activity Question and Compliance Checklists to confirm Annex implementation and capability for Safety Oversight Phase 2. On-site audit to confirm information and assessment of State’s overall capability for Safety Oversight The USAOP is based on a comprehensive systems approach using two phases. The first phase is an assessment of the implementation of the annex provisions and a review of differences through an advance review using the State Aviation Activity Questionnaire and Compliance Checklists submitted by the state. The Second Phase of the audit is the on site visit by the audit team to validate the information provided by the state and to assess the state’s overall capability for safety oversight. This focuses on a State’s organization, processes, procedures and programmes established and maintained by the state to meet its safety obligations RDIMS/SGDDI

Foundation Documents Rules of Conduct Memorandum of Understanding Audit documentation Pre-audit tools - State Aviation Activity Questionnaire - Compliance Checklist Protocol Checklists - On site interview questions o       The foundation documents for this program address: 1. Rules of conduct through the Memorandum of Agreement; and, 2. The Audit documentation through: The Pre-audit documentation: - the State Aviation Activity Questionnaire; and - the Compliance Checklist; and The on site audit tool; - the Protocol Checklists, used to standardize the on site interview questions. RDIMS/SGDDI

Memorandum of Understanding Foundation Documents cont’d Memorandum of Understanding Defines obligations for the audit Sent to State with notification of ICAO’s scheduled intention - 9 months in advance of audit date States are expected to sign and return 90 days before the audit The obligations of both ICAO and the State are defined in the Memorandum of Understanding. This document is submitted to the State about 9 months in advance of the planned audit date along with ICAO’s notification of intent to audit. The state is expected to sign the document and return it to ICAO at least 90 days prior to the audit. RDIMS/SGDDI

Foundation Documents cont’d MOU Highlights Scope to address all annexes except 9 and 17 ICAO team members will have command of English and at least one member will have command of the language chosen by the State Audit will be conducted in accordance with “Safety Oversight Audit Manual Doc 9735) ICAO to conduct the audit in accordance within a specified timeline In this agreement , ICAO commits to: Defining the scope of the audit to address all annexes except 9 and 17 (Facilitation and Security); Providing an audit team with a command of English and at least one member to have command of the language chosen by the State; Conducting the Audit in accordance with ICAO’s “Safety Oversight Audit Manual (Doc 9735); and Conducting the audit in accordance within a specified timeline. RDIMS/SGDDI

Foundation Documents cont’d State agrees: - to provide necessary access to staff, documents, state facilities; - to provide transportation within the state; - to provide required translation services and office facilities; - to submit updated version of SAAQ and CC at least 90 days before the audit - to establish a national coordinator; and - to assist with accommodation arrangements. The State will agree in signing this MOU: - to provide access to staff, documents, and state facilities; - to provide transportation within the state; - to provide required translation services and office facilities; - to submit an updated version of SAAQ and CC at least 90 days before the audit; - to establish a national coordinator; and - to assist with accommodation arrangements. RDIMS/SGDDI

Documentation tools Preaudit Foundation Documents cont’d Documentation tools Preaudit State Aviation Activity Questionnaire Compliance Checklists - Forms are available on ICAO web site To obtain information in advance of an audit, ICAO uses two Forms: the State Aviation Activity Questionnaire and the Compliance Checklist. These forms are available on the ICAO web site and are designed to provide the auditors with advance planning information on subjects ranging from state administration to regulatory compliance with ICAO’s Standards and Recommended Practices. RDIMS/SGDDI

State Aviation Activity Questionnaire Foundation Documents cont’d State Aviation Activity Questionnaire Advance information required by ICAO for scheduling and workload planning Provides overview information on State Activity organized into seven parts: General Administrative Legislative Organization Operational Activities Air Navigation Activities Aerodromes Accident and Incident Investigation The State Aviation Activity Questionnaire provides the ICAO auditor with information about the state administration and organization, including hours of work, special holidays, numbers of employees, client inventory and overview information on the State activity organized into 7 parts. The aim is to provide the audit team leader with information necessary to support planning for activities and team requirements. RDIMS/SGDDI

Compliance Checklists Advance information required by ICAO for assessing compliance with SARPs Presents State with tool to identify differences to SARPs Update required 90 days in advance of an audit An additional planning tool is the Compliance Checklist. Each state is required keep this form up to date on an annual basis as well provide an update 90 days in advance of an ICAO audit. The form is a listing of all the ICAO Standards, Recommend practices and guidance material for each annex with a requirement for the state to list their corresponding state references or provide supporting documentation to justify their differences. This in effect provides a state self assessment report that permits the auditor to identify areas of concern and potential follow-up during the audit. RDIMS/SGDDI

Foundation Documents cont’d Protocol Checklists A tool for conducting the on site audit or internal self assessment used to ensure: Transparency Consistency Standardization Organized to conduct critical element assessment in eight modules: Legislation State Organization Personnel Licensing Operation of Aircraft Airworthiness Accident Investigation Air Navigation Aerodromes The on-site audit is conducted using standardized questions called the “Protocol Checklist”. There is a protocol Checklist for each specialty area and these are available on ICAO’s web site for the state to review and organize themselves against, well before the audit. This advance information ensures standardization and consistency for all audits and supports a transparent process. RDIMS/SGDDI

Guidance Sources ICAO intranet website provides access to: State Aviation Activity Questionnaire Compliance Checklists Protocol Checklists ICAO Annexes and Documents Completed audits on member states ICAO Coordinators course Also listed on the web ICAO Team leader for the State audit Makes a State/ICAO team for meeting both goals Canada viewed the ICAO audit as a parallel program to our own auditing process and additional specific information and guidance for the ICAO program was found to be readily available. Sources included: The ICAO web site provides all the Questionnaires and checklists previously mentioned; The ICAO course for training the state coordinator. In Canada’s case, our state coordinator had already taken the ICAO auditors course instead; and Direct communication with ICAO’s Team lead to clarify any concerns as they arose. RDIMS/SGDDI

Audit timeline - On site activity Entry meeting Opportunity for introductions; ICAO provides explanation of process; and The State to provide a description of their organization. Conduct of interviews Start with group interviews for Legislation and Organization; Importance of time management - Audit plans are on strict schedule - State coordinators to ensure Protocol questions are organized, assigned and prepared for the interviews - ICAO team continues their work after State hours - A complex audit will take up to two weeks - Industry visits require State observer The Audit process has the normal audit stages of the Entry meeting, the Interview Stage and the Closing meeting. The Entry meeting accomplishes three objectives: It provides formal introductions of both the state and ICAO members; it allows ICAO to explain their audit process; and the state is expected to provide an overview description of their organization and regulatory program. The interview session begins began with a group session on the Legislative and Organizational elements of the State’s program and then the auditors proceed to their respective specialty interviews. The audit plan is on a strict schedule and time management is critical. After the interviews are complete each day, the team will work into the evenings discussing and documenting their observations so there is no surplus time available. RDIMS/SGDDI

Audit timeline - On site activity Closing meeting meeting is for ICAO team to present their preliminary findings no discussion of these findings is intended beginning of timeline for acceptance of findings and the final report. The closing meeting is the time for the ICAO auditors to present their findings. All discussions are to have been completed before this meeting and it is expected that questions will be limited to points of clarification. RDIMS/SGDDI

Audit Timeline - Post Audit 90 days 60 days Closing meeting interim report to State ICAO provides 30 days ICAO translates and posts on Web State clarifications to ICAO 30 days ICAO clarifies state response The timeline for the post audit reports then begins from the closing meeting as follows: ICAO has 90 days to provide an interim report for the state to review. After the report is received from ICAO, the State has 60 days to respond with their Corrective Action Plans. ICAO has 30 days to clarify State comments and their Action plan. ICAO will arrange for translation and publication on the web within the following 30 days. Note, unlike previous ICAO audit programs, there will be no follow-up inspections on the findings. Each state is expected to keep the status of their Corrective Action Plans up-to-date through the ICAO Web. RDIMS/SGDDI

Canadian Preparation for the Audit Coordination Communications Arrangements for daily debriefs Training for State interviewees Canada had volunteered as the “Beta” audit for the new ICAO process and there was really little time for advance preparation. In fact, ICAO only provided the last revisions to the Protocol questions on the Friday of the week before the audit. According to a follow-up survey with our participants though, this was not thought to have had an impact on the audit outcome. The ICAO process was consistent with our expectations for an audit; and, their own defined program. The result was no difficulty in understanding or complying with their requirements. Our state coordinator was appointed in Feb and planning for the April audit began immediately to address: Coordination; Communications; Including the requirement for daily debriefs; and Training for State interviewees. RDIMS/SGDDI

Application of ICAO process in The Pre-Audit phase Cont’d Coordination three areas to be involved in coordination – Civil Aviation, Transportation Safety Board and Department of National Defense Started work on advance arrangements for: Accommodations and office facilities Security clearances Transportation arrangements for industry and regional visits Signatures for the MOU was the first state coordination issue. This was due to the fact that not all the authorities to address the 16 annexes came under our Civil Aviation Authority. Both the Transport Safety Board and Department of National Defence also had to agree to support ICAO’s Audit process and timeframe. At the same time, work was begun on coordinating the logistics to support accommodations, office facilities, access clearances, transportation support and industry participation. RDIMS/SGDDI

Need for Internal Coordination System Scope for Canadian auditing initiatives contin’d Need for Internal Coordination System - State organizational structure does not parallel audit protocol organization or ICAO annexes - need to arrange and schedule appropriate interview candidates - need to support ICAO auditor during the audit As well, there was a need to develop and maintain an internal coordination system for the audit within our Directorate. ICAO’s audit protocol was organized in line with the organization of its Annexes but Canada’s Civil Aviation organization does not parallel this structure. Therefore a plan had to be developed to ensure that, for each Functional Branch within the Canadian Civil Aviation structure, there was one identified responsible contact to support coordination requirements. The responsibility for each Annex was assigned to a Functional Branch and, in cases where an Annex related to two Branches, the Branch with the primary involvement was assigned as the coordination role for inputs for other Branches. The coordinators for each Branch also assumed the role of arranging internal interview schedules and ensuring that the auditors were escorted to and from a designated rendezvous point to their arranged interview locations. During the audit, these key internal figures provided the added benefit of interview flexibility and support by arranging for interviews with supplemental expertise and making schedule adjustments. RDIMS/SGDDI

Communications Need for close coordination between State Coordinator Application of ICAO process in The Pre-Audit phase Cont’d Communications Need for close coordination between State Coordinator and ICAO Team Resulted in effective coordination for planning and resolution of Protocol question issues Introduction of Daily Audit debrief requirement Daily feedback for Management and Interviewees Triggers for providing follow-up clarification ICAO tool for managing “areas of concern” Essential for all phases of the audit from the MOU stage in the beginning, through to the final report clarifications, was the requirement for good communications between the ICAO Team lead and the State Coordinator. This resulted in excellent coordination with the audit team, effective advance planning and resolution of some Protocol questions issues before the audit began. Our advance planning also made arrangements to continue this dialogue through “Daily Debriefs” between the audit Team leader and our State Coordinator during the audit. These sessions were held at the end of each interview day, where the ICAO team lead provided a list of their arising “Areas of Concern” and a record of these observations were distributed to: the ICAO team to confirm our understanding of what had been said; the respective Branch coordinators to provide feedback on need for clarification from the interviews to remove any potential misunderstandings; and the Branch managers to appreciate the emerging issues. ICAO also found the process as a useful tool for managing their “Areas of Concern” during their onsite audit program and committed to continuing this step in their future audits. RDIMS/SGDDI

Training for the State’s interviewees Scope for Canadian auditing initiatives contin’d Training for the State’s interviewees - Improved understanding of their role and the audit process; - Guidance on role reversal from auditor to interviewee; and - Guidance on what are suitable questions. The ICAO audit had a psychological aspect that had to be addressed for our inspectors to ensure the process went smoothly. State staff are familiar with the “authority” role during an audit but needed to be conditioned to accept the interviewee role. To facilitate this, background information was provided on the upcoming ICAO audit and guidance was given to the interviewees regarding their behavior in this role reversal. This included advising them to limit their answers to the specific question and to the situation during the time of the audit, not what is being planned; not to argue with the auditors; and to limit responses to their area of expertise. This last item was difficult because, by nature most people try to be cooperative, but comments outside a mandated area of expertise could and did lead to potential confusion. RDIMS/SGDDI

Preparation by the Interviewee Protocol questions for specialty areas are available on ICAO’s web site; Critical elements numbers on the protocol questions confused the interviewees during their preparation for the audit The most effective interviews were the ones where the interviewees had reviewed the questions on the Protocol forms from ICAO’s web site and were prepared in advance. This avoided searching for reference material during the interview and lead some interviewees to come to their interviews with formal written responses to support their oral answers. During this preparation, some interviewees were confused by the Critical Element numbers in right hand column of the Protocol form. It was intended for use by the auditors to organize their report material but presented some confusion which was not identified until after the audit. Training and practice with the forms themselves, in hindsight, would have been useful. RDIMS/SGDDI

Observations on the Canadian Audit Experience - description of ICAO’s Audit Plan; - observations on the interviews; - observations on the audit process; and - a description of the Findings This next section is included to provide, from the Canadian experience, some expectations of what happens during the audit. It will cover: - description of ICAO’s Audit Plan; - observations on the interviews; - observations on the audit process; and - a description of the Findings. RDIMS/SGDDI

Page from ICAO Work Plan for OPS Specialist The audit on Canada is an example of a more complex audit conducted by ICAO. Such audits are expected to take about two weeks while others may be done in less than one week. ICAO’s actual plan is provided in a format which shows the planned activities for each specialty area separately. This slide shows the plan for the Operations specialist but there was no overall plan provided. RDIMS/SGDDI

A Description of ICAO’s Audit Plan This next slide is our compilation of these specialist plans to present the overall audit. The first Monday was used by the Team to travel to Ottawa and the Opening meeting began on Tuesday morning. It was attended by the complete ICAO audit team and all involved T.C. staff. As many involved T.C. staff as possible were invited to attend to foster an understanding and commitment for the process. The entire ICAO team conducted their Legislative and Organization interviews as a group over the next two, half day sessions and then the auditors proceeded individually to their specialty interviews. The specialty interviews began in Headquarters and then proceeded to two regional offices to include “Industry Site Visits”. Although there are no Protocol questions for the “Industry Site Visits,” they are part of the audit and are conducted as thoroughly as the interviews. During these sessions, the goal is to verify examples of the State fulfilling its authorization and oversight responsibilities and the auditors use probing questions to accomplish this task. A State specialist is expected to accompany the auditor and qualified experts from the host industries are required to answer the auditor’s questions. The later part of the second week was used by the auditors to follow-up on issues they wanted to investigate further and for a closed meeting among themselves to discuss their findings and prepare for the closing meeting. RDIMS/SGDDI

Observations on the Interviews The Interview Process Contin’d Observations on the Interviews ICAO’s lack of organizational understanding lead to misdirection of questions and inappropriate responses Interviewees were frustrated by auditors changing the order of the Protocol questions as well as by questioning from shaded areas - T.C. frustration over amount of inputs that were not addressed No use of non-international inventories Did not address Annex 10 The most difficulty during the interviews pertained to a lack of understanding of our State’s organization. Misunderstandings of T.C’s organizational structure lead to misdirection of some of the audit questions and this was compounded by the effort of our staff to cooperate in providing answers to the questions directed to them. Aside from this problem which was addressed through the Daily Feedback sessions, other minor frustrations expressed after the audit included: The interviewers changing the order of their protocol questions; The amount of inputs that were not addressed in the audit. Examples included material provided on domestic operators and the Compliance Checklists for Annex 10; and The interviewees were surprised to encounter questions from the shaded areas of the protocol list; RDIMS/SGDDI

Daily Debriefs Began on the end of second full day of interviews Attended by State coordinator and ICAO Team Lead Records provided to coordinators and ICAO Interviewees verified proper understanding and necessity to provide additional information ICAO used to manage their follow-up issues and confirm accuracy of record During the Daily Debriefs, ICAO used the terminology “Area of concern” and at the closing meeting the term was changed to “Preliminary Draft Finding” The Daily Debriefs proved to be effective. They began on the end of the second full day of interviews and were attended by the State Coordinator and ICAO team lead. The discussions were documented by the State coordinator staff and distributed to the interviewees to verify the necessity for our Branches to provide additional information for clarification. A copy was also given to the ICAO audit team to manage their follow-up issues and confirm accuracy of record. In these meetings, the term “Finding” was not used. ICAO observations were raised as “areas of concern” and may or may not have evolved into “Preliminary Draft Findings” at the closing meeting. RDIMS/SGDDI

The closing meeting Canada had all audit participants attend to ensure full understanding of results and to undertake immediate attention to corrective action. Each ICAO specialist summarized the preliminary findings for their area Canada responded with commitment to follow through on Findings. ICAO ended the meeting with the request to complete a State Assessment form and the commitment to continue using the “Daily Debrief” as part of their audit process The closing meeting followed the expected format: Canada had all its audit participants attend to ensure full understanding of results and to begin immediate work on corrective action responses; Each ICAO specialist summarized the preliminary findings for their specialty area; Canada responded with a commitment to follow through on the Findings; and ICAO ended the meeting with the request to complete a State Assessment form and their commitment to continue using the “Daily Debrief” as part of their audit process. RDIMS/SGDDI

Closing meeting Contin’d Was the audit a success? From the state perspective; Were the logistics met? From the State and ICAO perspective; Was a proper understanding of the state program developed to permit a satisfactory “systems” assessment? The question of “Was the audit a success?” leads to the question of how can we measure success of an audit? There will always be findings and opportunities for improvement. So the actual audit’s success must be measured in other terms. From the state perspective; - Were the logistics met? The audit did progress on time and not one auditor was lost. From the State and ICAO perspective; - Was a proper understanding of the state program developed to permit a satisfactory “systems” assessment? From that perspective, both have reached an agreement on the final report and an action plan is in place to address the outstanding items. RDIMS/SGDDI

Findings ICAO auditors admitted that they had to dig deep to find non-compliance There were “areas of concern” we disagreed with and upon clarification were able to resolve Others included: international conditions on licenses Cabin crew flight and duty times PPCs every two years Notify ICAO of exemptions But no matter how much emphasis we say we do or don’t put on Findings, the success of an audit does get judged by the number and content of the resulting Findings. The ICAO auditors say they had to dig deeply to find non compliances and we were able to clarify some areas of concern to prevent erroneous findings. There were findings though, and we are committed to correcting them. Examples include: - international conditions on licenses; - Cabin crew flight and duty times; - PPCs every two years; and - Notifying ICAO of exemptions. RDIMS/SGDDI

Lessons Learned Being a “Beta” audit did not impact the final results A shortage of time did not prevent preparation for the audit Staff wanted more information on ICAO and its requirements. Feedback was for more information on ICAO role, terminology and a simplified version of the audit process Need for state to adopt more proactive role in managing interviews Ensure assignment of Interviewers to appropriate expertise in the organization Promote requirement for advance preparation of questions From our Canadian experience, a list of lessons learned has been developed: Being a “Beta” audit did not impact the final results. The shortage of time did not prevent adequate preparation for the audit; Staff wanted more information on ICAO and its requirements. Feedback was for more information on ICAO role, terminology and a simplified version of the audit process. Presumably, better understanding should translate to better commitment; Need for state to adopt more proactive role in managing interviews. Ensure assignment of Interviewers to appropriate expertise in the organization; and Promote requirement for advance preparation of questions. RDIMS/SGDDI

Lessons Learned Contin’d Lessons Learned (continued) State response to “areas of concern” required specific focus Too much material prevented clear understanding of the state position Communication plan was essential Resulted in team building and knowledgeable inputs Daily debrief supported clear understanding by all participants Prevented development of inappropriate “areas of concern” 4. Too much material prevented clear understanding of the state position while attempting to clarify “areas of Concern”. Considering the limited time available, auditors can not be expected to consider responses and material outside that which clearly focuses on their questions. 5. The communications plan was essential. It resulted in: - Team building and knowledgeable inputs; - The Daily debrief providing clear understanding of interview results by all; and prevented development of inappropriate “areas of concern” RDIMS/SGDDI

Pitfalls in Advance of an Audit Not using all the resources available to understand and prepare for an audit. - ICAO web, ICAO team leader, State auditing expertise Allowing communication barriers - Ensure staff have clear roles that are understood in the program - Test the process for understanding and effectiveness Pitfalls in advance of the audit include: Not using all the resources available to understand and prepare for an audit. Options include ICAO web, ICAO team leader and our expertise with the auditing process. And; Allowing communication barriers: Ensure staff have clear roles and understanding of the process; Or conversely,test the process for understanding and effectiveness. RDIMS/SGDDI

3. Unprepared interviewees Pitfalls in Advance of an Audit Contin’d 3. Unprepared interviewees Provide training on process and expected attitude Interviewees to understand the limit of their responsibility to answer questions Don’t assume interviewees have knowledge of the process Prepare in advance, formal responses to the protocol questions and be prepared with supporting reference documentation Address apparent regulatory or organizational protocol issues with the time remaining before the actual audit 3. Unprepared interviewees - Provide training on process and expected attitude: - Interviewees must understand the limit of their responsibility/capability to answer questions; - Don’t assume interviewees have knowledge of the process; Training is essential; - Prepare formal responses to the protocol questions in advance and be prepared with supporting reference documentation - Fix things before the audit. Use the protocol questions to identify regulatory or organizational protocol issues with the time remaining before the actual audit and work on fixing them. If nothing else is accomplished , you will already have the Corrective Action Plan in place for after the audit. RDIMS/SGDDI

Potential Pitfalls During an Audit 1. Misunderstandings ICAO understanding of state organization impacts their protocol management and accuracy of concerns State members must understand the limits of their interview responsibilities Purpose of industry visits Pitfalls during an audit include: 1. Misunderstandings ICAO’s understanding of a state’s organization impacts their protocol management and accuracy of concerns. Questions must be directed to the appropriate functional specialist. Or restated another way, State members must understand the limits of their interview responsibilities and not to answer questions outside their assigned knowledge area even if they think they can. Purpose of industry visits. The term visit implies something less than an audit and to think that can lead to trouble. Visits are part of the audit and the auditor wants to speak to knowledgeable staff to confirm state activities by observing examples. RDIMS/SGDDI

Potential Pitfalls During an Audit Contin’d Giving too much information. Know when to stop and how to focus the material to the key point that will be understood Not using the “Daily Debrief” system, the feedback pulse for the audit Accepting General Findings based on limited interviews 2. Giving too much information. Know when to stop and how to focus the material to the key point that will be understood. Offering more than is required could open up new areas for Concern. 3. Not using the “Daily Debrief” system. This is the feedback pulse for the audit and must be used to ensure the auditors leave with the proper understanding from the interviews. 4. Accepting General Findings based on limited interviews. Make sure that the conclusions or observations are not based on isolated evidence. RDIMS/SGDDI

Summary Strong coordinator essential Send coordinator on ICAO training Select knowledgeable Branch coordinators Short and direct answers to questions Establish good communications with ICAO team leader Prepare interviewees including industry 1.Strong coordinator essential 2.Send coordinator on ICAO training 3.Select knowledgeable Branch coordinators 4.Short and direct answers to questions 5.Establish good communications with ICAO team leader 6.Prepare interviewees including industry RDIMS/SGDDI

Summary (continued) 7. Insist on daily debriefs 8. Escort auditors everywhere 9. Plan to have staff available at all times during the audit including weekends and late at night 10. Work with Auditors to ensure “Areas of Concern” accurately reflect program 7. Insist on daily debriefs 8. Escort auditors everywhere 9. Plan to have staff available at all times during the audit including weekends and late at night 10. Work with Auditors to ensure “Areas of Concern” accurately reflect program RDIMS/SGDDI

Questions? RDIMS/SGDDI