1. The Canadian USOAP Experience

2. The Canadian USOAP Experience
Aim:
to share Canadian experience gained from the ICAO
audit process to provide other states with an insight that
may be helpful in preparing for upcoming audits
Outline
Overview of the ICAO audit process
Canadian preparation
Canada’s Audit experience
Lessons Learned
Pitfalls to avoid
Link:
Canada was the first state to be audited by ICAO’s Universal Safety Oversight Audit Program in the new expanded form addressing 16 of the 18 Annexes and this presentation is being provided to describe our experiences with that process.
The Aim is to share our experiences gained through this process to provide other states with an insight that may be helpful in preparing for your upcoming audits.
To accomplish this, the format of this presentation will include a description of:
- an overview on the ICAO audit process;
- Canada’s preparation for the audit;
- the actual audit experience;
- and a summary of the lessons learned and pitfalls to avoid
RDIMS/SGDDI

3. Overview of the ICAO audit process
Program Objective (Assembly Resolution A35-6)
Assess State’s implementation of:
- Safety oversight, and
- Standards and Recommended Practices, associated procedures and guidance material
ICAO’s Universal Safety Oversight program evolved from the voluntary role beginning in 1995 addressing the 3 annexes, 1,6 and 8, to their current mandatory program addressing the 16 Safety related Annexes. Their program objective is to promote global aviation safety by auditing contracting states to determine the status of a State’s implementation of:
Safety Oversight obligations and;
the relevant Standards and Recommended Practices, associated procedures and guidance material.
The words Recommended Practices and Guidance Material on this slide are underlined to draw attention to the fact the Assembly Resolution has mandated their program to audit beyond the Standards requirement to include material that some states may not view as an ICAO commitment.
RDIMS/SGDDI

4. ICAO auditing Philosophy cont’d
ICAO’s Comprehensive Systems approach:
Phase 1. Review of State Aviation Activity Question and Compliance Checklists to confirm Annex implementation and capability for Safety Oversight
Phase 2. On-site audit to confirm information and assessment of State’s overall capability for Safety Oversight
The USAOP is based on a comprehensive systems approach using two phases.
The first phase is an assessment of the implementation of the annex provisions and a review of differences through an advance review using the State Aviation Activity Questionnaire and Compliance Checklists submitted by the state.
The Second Phase of the audit is the on site visit by the audit team to validate the information provided by the state and to assess the state’s overall capability for safety oversight. This focuses on a State’s organization, processes, procedures and programmes established and maintained by the state to meet its safety obligations
RDIMS/SGDDI

5. Foundation Documents Rules of Conduct Memorandum of Understanding
Audit documentation
Pre-audit tools
- State Aviation Activity Questionnaire
- Compliance Checklist
Protocol Checklists
- On site interview questions
o      
The foundation documents for this program address:
1. Rules of conduct through the Memorandum of Agreement; and,
2. The Audit documentation through:
The Pre-audit documentation:
- the State Aviation Activity Questionnaire; and
- the Compliance Checklist; and
The on site audit tool;
- the Protocol Checklists, used to standardize the on site interview questions.
RDIMS/SGDDI

6. Memorandum of Understanding
Foundation Documents cont’d
Memorandum of Understanding
Defines obligations for the audit
Sent to State with notification of ICAO’s scheduled intention
- 9 months in advance of audit date
States are expected to sign and return 90 days before the audit
The obligations of both ICAO and the State are defined in the Memorandum of Understanding. This document is submitted to the State about 9 months in advance of the planned audit date along with ICAO’s notification of intent to audit. The state is expected to sign the document and return it to ICAO at least 90 days prior to the audit.
RDIMS/SGDDI

7. Foundation Documents cont’d
MOU Highlights
Scope to address all annexes except 9 and 17
ICAO team members will have command of English and
at least one member will have command of the language chosen
by the State
Audit will be conducted in accordance with “Safety Oversight
Audit Manual Doc 9735)
ICAO to conduct the audit in accordance within a specified
timeline
In this agreement , ICAO commits to:
Defining the scope of the audit to address all annexes except 9 and 17 (Facilitation and Security);
Providing an audit team with a command of English and
at least one member to have command of the language chosen
by the State;
Conducting the Audit in accordance with ICAO’s “Safety Oversight Audit Manual (Doc 9735); and
Conducting the audit in accordance within a specified timeline.
RDIMS/SGDDI

8. Foundation Documents cont’d
State agrees:
- to provide necessary access to staff, documents,
state facilities;
- to provide transportation within the state;
- to provide required translation services
and office facilities;
- to submit updated version of SAAQ and
CC at least 90 days before the audit
- to establish a national coordinator; and
- to assist with accommodation arrangements.
The State will agree in signing this MOU:
- to provide access to staff, documents, and state facilities;
- to provide transportation within the state;
- to provide required translation services and office facilities;
- to submit an updated version of SAAQ and CC at least 90 days before the audit;
- to establish a national coordinator; and
- to assist with accommodation arrangements.
RDIMS/SGDDI

9. Documentation tools Preaudit
Foundation Documents cont’d
Documentation tools Preaudit
State Aviation Activity Questionnaire
Compliance Checklists
- Forms are available on ICAO web site
To obtain information in advance of an audit, ICAO uses two Forms: the State Aviation Activity Questionnaire and the Compliance Checklist. These forms are available on the ICAO web site and are designed to provide the auditors with advance planning information on subjects ranging from state administration to regulatory compliance with ICAO’s Standards and Recommended Practices.
RDIMS/SGDDI

10. State Aviation Activity Questionnaire
Foundation Documents cont’d
State Aviation Activity Questionnaire
Advance information required by ICAO for scheduling
and workload planning
Provides overview information on State Activity
organized into seven parts:
General Administrative
Legislative
Organization
Operational Activities
Air Navigation Activities
Aerodromes
Accident and Incident Investigation
The State Aviation Activity Questionnaire provides the ICAO auditor with information about the state administration and organization, including hours of work, special holidays, numbers of employees, client inventory and overview information on the State activity organized into 7 parts. The aim is to provide the audit team leader with information necessary to support planning for activities and team requirements.
RDIMS/SGDDI

11. Compliance Checklists
Advance information required by ICAO for assessing compliance with SARPs
Presents State with tool to identify differences to SARPs
Update required 90 days in advance of an audit
An additional planning tool is the Compliance Checklist. Each state is required keep this form up to date on an annual basis as well provide an update 90 days in advance of an ICAO audit. The form is a listing of all the ICAO Standards, Recommend practices and guidance material for each annex with a requirement for the state to list their corresponding state references or provide supporting documentation to justify their differences.
This in effect provides a state self assessment report that permits the auditor to identify areas of concern and potential follow-up during the audit.
RDIMS/SGDDI

12. Foundation Documents cont’d
Protocol Checklists
A tool for conducting the on site audit or internal self assessment used to ensure:
Transparency
Consistency
Standardization
Organized to conduct critical element assessment
in eight modules:
Legislation
State Organization
Personnel Licensing
Operation of Aircraft
Airworthiness
Accident Investigation
Air Navigation
Aerodromes
The on-site audit is conducted using standardized questions called the “Protocol Checklist”. There is a protocol Checklist for each specialty area and these are available on ICAO’s web site for the state to review and organize themselves against, well before the audit. This advance information ensures standardization and consistency for all audits and supports a transparent process.
RDIMS/SGDDI

13. Guidance Sources ICAO intranet website provides access to:
State Aviation Activity Questionnaire
Compliance Checklists
Protocol Checklists
ICAO Annexes and Documents
Completed audits on member states
ICAO Coordinators course
Also listed on the web
ICAO Team leader for the State audit
Makes a State/ICAO team for meeting both goals
Canada viewed the ICAO audit as a parallel program to our own auditing process and additional specific information and guidance for the ICAO program was found to be readily available. Sources included:
The ICAO web site provides all the Questionnaires and checklists previously mentioned;
The ICAO course for training the state coordinator. In Canada’s case, our state coordinator had already taken the ICAO auditors course instead; and
Direct communication with ICAO’s Team lead to clarify any concerns as they arose.
RDIMS/SGDDI

14. Audit timeline - On site activity
Entry meeting
Opportunity for introductions;
ICAO provides explanation of process; and
The State to provide a description of their organization.
Conduct of interviews
Start with group interviews for Legislation
and Organization;
Importance of time management
- Audit plans are on strict schedule
- State coordinators to ensure Protocol questions are
organized, assigned and prepared for the interviews
- ICAO team continues their work after State hours
- A complex audit will take up to two weeks
- Industry visits require State observer
The Audit process has the normal audit stages of the Entry meeting, the Interview Stage and the Closing meeting.
The Entry meeting accomplishes three objectives: It provides formal introductions of both the state and ICAO members; it allows ICAO to explain their audit process; and the state is expected to provide an overview description of their organization and regulatory program.
The interview session begins began with a group session on the Legislative and Organizational elements of the State’s program and then the auditors proceed to their respective specialty interviews.
The audit plan is on a strict schedule and time management is critical. After the interviews are complete each day, the team will work into the evenings discussing and documenting their observations so there is no surplus time available.
RDIMS/SGDDI

15. Audit timeline - On site activity
Closing meeting
meeting is for ICAO team to present
their preliminary findings
no discussion of these findings is intended
beginning of timeline for acceptance of findings
and the final report.
The closing meeting is the time for the ICAO auditors to present their findings. All discussions are to have been completed before this meeting and it is expected that questions will be limited to points of clarification.
RDIMS/SGDDI

16. Audit Timeline - Post Audit
90 days
60 days
Closing meeting
interim report to State
ICAO provides
30 days
ICAO translates and posts on Web
State clarifications
to ICAO
30 days
ICAO clarifies state response
The timeline for the post audit reports then begins from the closing meeting as follows:
ICAO has 90 days to provide an interim report for the state to review.
After the report is received from ICAO, the State has 60 days to respond with their Corrective Action Plans.
ICAO has 30 days to clarify State comments and their Action plan.
ICAO will arrange for translation and publication on the web within the following 30 days.
Note, unlike previous ICAO audit programs, there will be no follow-up inspections on the findings. Each state is expected to keep the status of their Corrective Action Plans up-to-date through the ICAO Web.
RDIMS/SGDDI

17. Canadian Preparation for the Audit
Coordination
Communications
Arrangements for daily debriefs
Training for State interviewees
Canada had volunteered as the “Beta” audit for the new ICAO process and there was really little time for advance preparation. In fact, ICAO only provided the last revisions to the Protocol questions on the Friday of the week before the audit. According to a follow-up survey with our participants though, this was not thought to have had an impact on the audit outcome.
The ICAO process was consistent with our expectations for an audit; and, their own defined program. The result was no difficulty in understanding or complying with their requirements. Our state coordinator was appointed in Feb and planning for the April audit began immediately to address:
Coordination;
Communications;
Including the requirement for daily debriefs; and
Training for State interviewees.
RDIMS/SGDDI

18. Application of ICAO process in The Pre-Audit phase Cont’d
Coordination
three areas to be involved in coordination – Civil Aviation, Transportation Safety Board and Department of National Defense
Started work on advance arrangements for:
Accommodations and office facilities
Security clearances
Transportation arrangements for industry and regional visits
Signatures for the MOU was the first state coordination issue. This was due to the fact that not all the authorities to address the 16 annexes came under our Civil Aviation Authority. Both the Transport Safety Board and Department of National Defence also had to agree to support ICAO’s Audit process and timeframe.
At the same time, work was begun on coordinating the logistics to support accommodations, office facilities, access clearances, transportation support and industry participation.
RDIMS/SGDDI

19. Need for Internal Coordination System
Scope for Canadian auditing initiatives contin’d
Need for Internal Coordination System
- State organizational structure does not parallel audit protocol organization or ICAO annexes
- need to arrange and schedule appropriate interview candidates
- need to support ICAO auditor during the audit
As well, there was a need to develop and maintain an internal coordination system for the audit within our Directorate. ICAO’s audit protocol was organized in line with the organization of its Annexes but Canada’s Civil Aviation organization does not parallel this structure. Therefore a plan had to be developed to ensure that, for each Functional Branch within the Canadian Civil Aviation structure, there was one identified responsible contact to support coordination requirements.
The responsibility for each Annex was assigned to a Functional Branch and, in cases where an Annex related to two Branches, the Branch with the primary involvement was assigned as the coordination role for inputs for other Branches. The coordinators for each Branch also assumed the role of arranging internal interview schedules and ensuring that the auditors were escorted to and from a designated rendezvous point to their arranged interview locations. During the audit, these key internal figures provided the added benefit of interview flexibility and support by arranging for interviews with supplemental expertise and making schedule adjustments.
RDIMS/SGDDI

20. Communications Need for close coordination between State Coordinator
Application of ICAO process in The Pre-Audit phase Cont’d
Communications
Need for close coordination between State Coordinator
and ICAO Team
Resulted in effective coordination for planning and
resolution of Protocol question issues
Introduction of Daily Audit debrief requirement
Daily feedback for Management and Interviewees
Triggers for providing follow-up clarification
ICAO tool for managing “areas of concern”
Essential for all phases of the audit from the MOU stage in the beginning, through to the final report clarifications, was the requirement for good communications between the ICAO Team lead and the State Coordinator. This resulted in excellent coordination with the audit team, effective advance planning and resolution of some Protocol questions issues before the audit began.
Our advance planning also made arrangements to continue this dialogue through “Daily Debriefs” between the audit Team leader and our State Coordinator during the audit. These sessions were held at the end of each interview day, where the ICAO team lead provided a list of their arising “Areas of Concern” and a record of these observations were distributed to:
the ICAO team to confirm our understanding of what had been said;
the respective Branch coordinators to provide feedback on need for clarification from the interviews to remove any potential misunderstandings; and
the Branch managers to appreciate the emerging issues.
ICAO also found the process as a useful tool for managing their “Areas of Concern” during their onsite audit program and committed to continuing this step in their future audits.
RDIMS/SGDDI

21. Training for the State’s interviewees
Scope for Canadian auditing initiatives contin’d
Training for the State’s interviewees
- Improved understanding of their role and the audit process;
- Guidance on role reversal from auditor to interviewee; and
- Guidance on what are suitable questions.
The ICAO audit had a psychological aspect that had to be addressed for our inspectors to ensure the process went smoothly. State staff are familiar with the “authority” role during an audit but needed to be conditioned to accept the interviewee role.
To facilitate this, background information was provided on the upcoming ICAO audit and guidance was given to the interviewees regarding their behavior in this role reversal. This included advising them to limit their answers to the specific question and to the situation during the time of the audit, not what is being planned; not to argue with the auditors; and to limit responses to their area of expertise. This last item was difficult because, by nature most people try to be cooperative, but comments outside a mandated area of expertise could and did lead to potential confusion.
RDIMS/SGDDI

22. Preparation by the Interviewee
Protocol questions for specialty areas are available on ICAO’s web site;
Critical elements numbers on the protocol questions confused the interviewees during their preparation for the audit
The most effective interviews were the ones where the interviewees had reviewed the questions on the Protocol forms from ICAO’s web site and were prepared in advance. This avoided searching for reference material during the interview and lead some interviewees to come to their interviews with formal written responses to support their oral answers.
During this preparation, some interviewees were confused by the Critical Element numbers in right hand column of the Protocol form. It was intended for use by the auditors to organize their report material but presented some confusion which was not identified until after the audit. Training and practice with the forms themselves, in hindsight, would have been useful.
RDIMS/SGDDI

23. Observations on the Canadian Audit Experience
- description of ICAO’s Audit Plan;
- observations on the interviews;
- observations on the audit process; and
- a description of the Findings
This next section is included to provide, from the Canadian experience, some expectations of what happens during the audit. It will cover:
- description of ICAO’s Audit Plan;
- observations on the interviews;
- observations on the audit process; and
- a description of the Findings.
RDIMS/SGDDI

24. Page from ICAO Work Plan for OPS Specialist
The audit on Canada is an example of a more complex audit conducted by ICAO. Such audits are expected to take about two weeks while others may be done in less than one week.
ICAO’s actual plan is provided in a format which shows the planned activities for each specialty area separately. This slide shows the plan for the Operations specialist but there was no overall plan provided.
RDIMS/SGDDI

25. A Description of ICAO’s Audit Plan
This next slide is our compilation of these specialist plans to present the overall audit. The first Monday was used by the Team to travel to Ottawa and the Opening meeting began on Tuesday morning. It was attended by the complete ICAO audit team and all involved T.C. staff. As many involved T.C. staff as possible were invited to attend to foster an understanding and commitment for the process.
The entire ICAO team conducted their Legislative and Organization interviews as a group over the next two, half day sessions and then the auditors proceeded individually to their specialty interviews. The specialty interviews began in Headquarters and then proceeded to two regional offices to include “Industry Site Visits”.
Although there are no Protocol questions for the “Industry Site Visits,” they are part of the audit and are conducted as thoroughly as the interviews. During these sessions, the goal is to verify examples of the State fulfilling its authorization and oversight responsibilities and the auditors use probing questions to accomplish this task. A State specialist is expected to accompany the auditor and qualified experts from the host industries are required to answer the auditor’s questions.
The later part of the second week was used by the auditors to follow-up on issues they wanted to investigate further and for a closed meeting among themselves to discuss their findings and prepare for the closing meeting.
RDIMS/SGDDI

26. Observations on the Interviews
The Interview Process Contin’d
Observations on the Interviews
ICAO’s lack of organizational understanding lead to misdirection of questions and inappropriate responses
Interviewees were frustrated by auditors changing the order of the Protocol questions as well as by questioning from shaded areas
- T.C. frustration over amount of inputs that were not addressed
No use of non-international inventories
Did not address Annex 10
The most difficulty during the interviews pertained to a lack of understanding of our State’s organization. Misunderstandings of T.C’s organizational structure lead to misdirection of some of the audit questions and this was compounded by the effort of our staff to cooperate in providing answers to the questions directed to them.
Aside from this problem which was addressed through the Daily Feedback sessions, other minor frustrations expressed after the audit included:
The interviewers changing the order of their protocol questions;
The amount of inputs that were not addressed in the audit. Examples included material provided on domestic operators and the Compliance Checklists for Annex 10; and
The interviewees were surprised to encounter questions from the shaded areas of the protocol list;
RDIMS/SGDDI

27. Daily Debriefs Began on the end of second full day of interviews
Attended by State coordinator and ICAO Team Lead
Records provided to coordinators and ICAO
Interviewees verified proper understanding and necessity to provide additional information
ICAO used to manage their follow-up issues and confirm accuracy of record
During the Daily Debriefs, ICAO used the terminology “Area of concern” and at the closing meeting the term was changed to “Preliminary Draft Finding”
The Daily Debriefs proved to be effective. They began on the end of the second full day of interviews and were attended by the State Coordinator and ICAO team lead.
The discussions were documented by the State coordinator staff and distributed to the interviewees to verify the necessity for our Branches to provide additional information for clarification. A copy was also given to the ICAO audit team to manage their follow-up issues and confirm accuracy of record.
In these meetings, the term “Finding” was not used. ICAO observations were raised as “areas of concern” and may or may not have evolved into “Preliminary Draft Findings” at the closing meeting.
RDIMS/SGDDI

28. The closing meeting
Canada had all audit participants attend to ensure full understanding of results and to undertake immediate attention to corrective action.
Each ICAO specialist summarized the preliminary findings for their area
Canada responded with commitment to follow through on Findings.
ICAO ended the meeting with the request to complete a State Assessment form and the commitment to continue using the “Daily Debrief” as part of their audit process
The closing meeting followed the expected format:
Canada had all its audit participants attend to ensure full understanding of results and to begin immediate work on corrective action responses;
Each ICAO specialist summarized the preliminary findings for their specialty area;
Canada responded with a commitment to follow through on the Findings; and
ICAO ended the meeting with the request to complete a State Assessment form and their commitment to continue using the “Daily Debrief” as part of their audit process.
RDIMS/SGDDI

29. Closing meeting Contin’d
Was the audit a success?
From the state perspective;
Were the logistics met?
From the State and ICAO perspective;
Was a proper understanding of the state program developed to permit a satisfactory “systems” assessment?
The question of “Was the audit a success?” leads to the question of how can we measure success of an audit? There will always be findings and opportunities for improvement. So the actual audit’s success must be measured in other terms.
From the state perspective;
- Were the logistics met? The audit did progress on time and not one auditor was lost.
From the State and ICAO perspective;
- Was a proper understanding of the state program developed to permit a satisfactory “systems” assessment? From that perspective, both have reached an agreement on the final report and an action plan is in place to address the outstanding items.
RDIMS/SGDDI

30. Findings
ICAO auditors admitted that they had to dig deep to find non-compliance
There were “areas of concern” we disagreed with and upon clarification were able to resolve
Others included:
international conditions on licenses
Cabin crew flight and duty times
PPCs every two years
Notify ICAO of exemptions
But no matter how much emphasis we say we do or don’t put on Findings, the success of an audit does get judged by the number and content of the resulting Findings. The ICAO auditors say they had to dig deeply to find non compliances and we were able to clarify some areas of concern to prevent erroneous findings.
There were findings though, and we are committed to correcting them. Examples include:
- international conditions on licenses;
- Cabin crew flight and duty times;
- PPCs every two years; and
- Notifying ICAO of exemptions.
RDIMS/SGDDI

31. Lessons Learned Being a “Beta” audit did not impact the final results
A shortage of time did not prevent preparation for the audit
Staff wanted more information on ICAO and its requirements. Feedback was for more information on ICAO role, terminology and a simplified version of the audit process
Need for state to adopt more proactive role in managing interviews
Ensure assignment of Interviewers to appropriate expertise in the organization
Promote requirement for advance preparation of questions
From our Canadian experience, a list of lessons learned has been developed:
Being a “Beta” audit did not impact the final results. The shortage of time did not prevent adequate preparation for the audit;
Staff wanted more information on ICAO and its requirements. Feedback was for more information on ICAO role, terminology and a simplified version of the audit process. Presumably, better understanding should translate to better commitment;
Need for state to adopt more proactive role in managing interviews.
Ensure assignment of Interviewers to appropriate expertise in the organization; and
Promote requirement for advance preparation of questions.
RDIMS/SGDDI

32. Lessons Learned Contin’d
Lessons Learned (continued)
State response to “areas of concern” required specific focus
Too much material prevented clear understanding of the state position
Communication plan was essential
Resulted in team building and knowledgeable inputs
Daily debrief supported clear understanding by all participants
Prevented development of inappropriate “areas of concern”
4. Too much material prevented clear understanding of the state position while attempting to clarify “areas of Concern”. Considering the limited time available, auditors can not be expected to consider responses and material outside that which clearly focuses on their questions.
5. The communications plan was essential. It resulted in:
- Team building and knowledgeable inputs;
- The Daily debrief providing clear understanding of interview results by all; and prevented development of inappropriate “areas of concern”
RDIMS/SGDDI

33. Pitfalls in Advance of an Audit
Not using all the resources available to understand and prepare for an audit.
- ICAO web, ICAO team leader, State auditing expertise
Allowing communication barriers
- Ensure staff have clear roles that are understood in the program
- Test the process for understanding and effectiveness
Pitfalls in advance of the audit include:
Not using all the resources available to understand and prepare for an audit. Options include ICAO web, ICAO team leader and our expertise with the auditing process. And;
Allowing communication barriers:
Ensure staff have clear roles and understanding of the process;
Or conversely,test the process for understanding and effectiveness.
RDIMS/SGDDI

34. 3. Unprepared interviewees
Pitfalls in Advance of an Audit Contin’d
3. Unprepared interviewees
Provide training on process and expected attitude
Interviewees to understand the limit of their responsibility to answer questions
Don’t assume interviewees have knowledge of the process
Prepare in advance, formal responses to the protocol questions and be prepared with supporting reference documentation
Address apparent regulatory or organizational protocol issues with the time remaining before the actual audit
3. Unprepared interviewees
- Provide training on process and expected attitude:
- Interviewees must understand the limit of their responsibility/capability to answer questions;
- Don’t assume interviewees have knowledge of the process; Training is essential;
- Prepare formal responses to the protocol questions in advance and be prepared with supporting reference documentation
- Fix things before the audit. Use the protocol questions to identify regulatory or organizational protocol issues with the time remaining before the actual audit and work on fixing them. If nothing else is accomplished , you will already have the Corrective Action Plan in place for after the audit.
RDIMS/SGDDI

35. Potential Pitfalls During an Audit
1. Misunderstandings
ICAO understanding of state organization impacts their protocol management and accuracy of concerns
State members must understand the limits of their interview responsibilities
Purpose of industry visits
Pitfalls during an audit include:
1. Misunderstandings
ICAO’s understanding of a state’s organization impacts their protocol management and accuracy of concerns. Questions must be directed to the appropriate functional specialist. Or restated another way, State members must understand the limits of their interview responsibilities and not to answer questions outside their assigned knowledge area even if they think they can.
Purpose of industry visits. The term visit implies something less than an audit and to think that can lead to trouble. Visits are part of the audit and the auditor wants to speak to knowledgeable staff to confirm state activities by observing examples.
RDIMS/SGDDI

36. Potential Pitfalls During an Audit Contin’d
Giving too much information.
Know when to stop and how to focus the material to the key point that will be understood
Not using the “Daily Debrief” system,
the feedback pulse for the audit
Accepting General Findings based on limited interviews
2. Giving too much information.
Know when to stop and how to focus the material to the key point that will be understood. Offering more than is required could open up new areas for Concern.
3. Not using the “Daily Debrief” system. This is the feedback pulse for the audit and must be used to ensure the auditors leave with the proper understanding from the interviews.
4. Accepting General Findings based on limited interviews. Make sure that the conclusions or observations are not based on isolated evidence.
RDIMS/SGDDI

37. Summary Strong coordinator essential Send coordinator on ICAO training
Select knowledgeable Branch coordinators
Short and direct answers to questions
Establish good communications with ICAO team leader
Prepare interviewees including industry
1.Strong coordinator essential
2.Send coordinator on ICAO training
3.Select knowledgeable Branch coordinators
4.Short and direct answers to questions
5.Establish good communications with ICAO team leader
6.Prepare interviewees including industry
RDIMS/SGDDI

38. Summary (continued) 7. Insist on daily debriefs
8. Escort auditors everywhere
9. Plan to have staff available at all times during the audit including weekends and late at night
10. Work with Auditors to ensure “Areas of Concern” accurately reflect program
7. Insist on daily debriefs
8. Escort auditors everywhere
9. Plan to have staff available at all times during the audit including weekends and late at night
10. Work with Auditors to ensure “Areas of Concern” accurately reflect program
RDIMS/SGDDI

39. Questions?
RDIMS/SGDDI