for Cabin Safety Inspectors Welcome to the Aviation Audit Course for Cabin Safety Inspectors CSI, Aviation Audit Course 1 First of all, I’d like to thank Barbara for the introduction…Thank you Barbara. I would like to welcome you all to this Aviation Audit Procedures Course for Cabin Safety Inspectors. Actually, even if you come from the other side of the fence (which is operator side), this course would still be a lot of help because it gives you a different perspective n what an audit is. 1

Objectives To gain better understanding of the purpose of an audit CSI, Aviation Audit Course Objectives To gain better understanding of the purpose of an audit To gain better techniques in auditing, which are in accordance with CAA standards 2 Our Course Objectives is straight forward...two of them...closely related 2

CSI, Aviation Audit Course AUDITING What is an audit suppose to accomplish?. … The auditing process should put the relationship between the company and CAA into balance. It enforces the requirement that the company is responsible for safety. If the company manages safety properly the company will not have any problems in addressing a CAA audit. 3 read To simplify these, what is it that we want from the industry? We just want the industry operators to ensure that everything is being thought of to ensure that we all do things all the time with safety in mind. We don’t want haphazard procedures, we don’t want tired and useless cabin crew ion the air, we want our passengers to reach their destination safely. 3

CSI, Aviation Audit Course AUDITING…cont... The company will not hide information about incidents and its failures because the company’s safety management team will accept and address the problems in the open. The company will also want to explain to CAA what went wrong and how the problem was rectified without fear. 4 read As you can see … an audit is suppose to improve the relationship between the Authority and the operator. It is suppose to open lines of communications so that the operator becomes more at home with the Authority and seeks advise before it tries something new and makes a mistake. 4

CSI, Aviation Audit Course What is an audit? An audit is an in-depth review of the activities of an organization to verify conformance to CAA regulations and standards. 5 It is impossible for me to over emphasize that an audit is in-depth review of all aspects of the organization ...approach should be one of complete openness promote conformance to regulatory requirements and automatically becomes synonymous to compliance with ICAO SARPS and protocols …it is therefore a series of factual statements outlining the overall level of conformance of the auditee 5

Overview of an Audit Process CSI, Aviation Audit Course Overview of an Audit Process 6 To fulfill your obligations as an inspector (or inspectee for industry participants) it is necessary that you be familiar with this process. We should always remember that the (NAP) National Audit Program was made to ensure that a satisfactory level of aviation safety is maintained, and that CAA audit policies and procedures are applied nationally in a uniform manner. 6

Outputs of an Audit Audit Report CAA Internal Report CSI, Aviation Audit Course Outputs of an Audit Audit Report CAA Internal Report 7 an Audit report is a series of factual statements outlining the overall level of conformance of the auditee the Parallel Report or the CAA Internal report consists of audit findings against ourselves...The Civil Aviation Authority....once again, we approach this with a positive attitude and objective viewpoint What is in an Audit? …next slide 7

BASIC ACTIVITIES OF AN AUDIT: CSI, Aviation Audit Course BASIC ACTIVITIES OF AN AUDIT: Pre-audit Physical audit Post audit Audit follow-up 8 What does this mean to us? 8

Audit Activity Breakdown CSI, Aviation Audit Course Audit Activity Breakdown Scope & Depth Team Selection Budget Documents Preps Team Meeting Entry Meeting Physical Insp.. Functional Areas Findings Daily Meetings Company De-brief Corrective Action Follow Up Audit Report Wrap-up Activities 9 The biggest part of an audit is planning Part of planning is knowing what are the things to be prepared for and what to look at. Each activity of the audit is graphed for us to be aware. Lets look at the 1st of these phase…the Pre-Audit phase …..next slide Pre-Audit Audit Post-Audit Follow-Up Start Finish 9

Pre- Audit CSI, Aviation Audit Course 26 10 10 Pre-Audit activities sets the tone for the entire audit. It is during this phase that credibility and respect for the audit management team can be established (or if not handled properly…lost, or at best, delayed). What are the pre-audit activities? ….next slide 26 10

Pre-Audit Activities Audit planning Team assembly CSI, Aviation Audit Course Pre-Audit Activities Audit planning Team assembly Pre-audit team meeting Final preparations including document review 11 You’ve all heard the phrase: “If you fail to plan you’re planning to fail”. The quality of an audit is directly proportional to the quality of planning...so plan the work...and work the plan. …part of Audit Planning – is knowing your schedule and duration for audit Distance location Unforeseen circumstances (weather, travel etc.) are but a small part of it 11

Audit Planning Rule of Thumb Small 1 - 5 days Medium 5 - 15 days CSI, Aviation Audit Course Audit Planning Rule of Thumb Small 1 - 5 days Medium 5 - 15 days Large 15 + days 12 We should bear in mind that some industry operators have about 75 aircrafts and not only one type 12

Scope CAA defines scope as the number of CSI, Aviation Audit Course Scope CAA defines scope as the number of areas within a company's operation that will be audited. 13 Scope is determined by the type or class of audit conducted……Combined, Specialist, Special Purpose It is also determined by: Size of company including personnel Records of company – qualification and training records, aircraft types and safety equipments and procedures, Flt time Review OC & OPS Specs - domestic, International operations, charter…? Type of aircrafts w/ Cabin crew Certificates of Approval, A/C limitations – does the operator do extended overwater operations? 13

CSI, Aviation Audit Course Depth The period of time over which a company’s operations will be audited, normally from the last audit until present day. It could also be thought of as the extent or the size of the audit. 14 Depending on what you see on the surface, with a little dusting, you need to go deeper. We need to always remember that we are not auditing the people but the system. We are looking for systemic not individual faults. 14

SAMPLING RATIONALE The audit procedures manual sets out CSI, Aviation Audit Course SAMPLING RATIONALE The audit procedures manual sets out a table for sampling to 95% confidence level S C I E N Confidence level of 95% is identified in the CAA guidelines. In practical terms it may not be realistic to review all aircraft in the fleet. Significant findings against one type of aircraft may require additional review against other types. You must be realistic when conducting the review and look at each audit situation independently. 15 Part of what to consider in the planning stage as to schedule is the sampling rationale. It is required that we must have 95% confidence level. So the more aircrafts, cabin crew, training records mean more time consumed in audit 15

Communication 1. Daily internal meetings with Team CSI, Aviation Audit Course Communication 1. Daily internal meetings with Team 2. External meeting with the Company 16 An essential part of being a member on an audit team is your ability to communicate…to both other members of the team, as well as the operator. It is important to advise the operator and your team of applicable lines of communication…Audit Managers should insist that company personnel respect and use the established communications channels. Internal communication include Team Meetings… To ensure the audit plan is followed; To ensure all team members understand with whom they will communicate; To ensure unforeseen problems are dealt with in a timely manner. Nothing will bring an audit “down” faster than poor communications…whereas the opposite is also true, Good communications result in effective, successful audits. - constant communication develops teamwork and shows supports for the group 16

Prep & Documentation review CSI, Aviation Audit Course Prep & Documentation review Outstanding previous audits Company Manuals CCSM Cabin Crew Training Program Surveillance reports since the previous audit Cabin enroute Pre-flight reports Proving reports Recurring findings and problems Monitored Trainings 17 For Previous audits… we try and determine what does the previous audit reveal? and Specify nature of the problem? … Is it a problem in training? Standards? ……are there any recurring problems? For Company manuals Cabin safety inspectors need to refer to the Cabin Crew Manual & Training Program job aid. Has it been updated in accordance with the latest revision? For surveillance reports we need to check on all reports of the past 12 months….. Have they been addressed by the operator? 17

Audit CSI, Aviation Audit Course 18 18 Lets now go to the audit itself… next slide 18

ENTRY MEETING Thank You's Explain the purpose of the meeting CSI, Aviation Audit Course ENTRY MEETING Topics to be covered Thank You's Explain the purpose of the meeting Introduction of Team & Company Rep Discuss objective and conduct of audit Discuss scope and depth Communications between the Audit Team & the company, including Daily Meetings Introduce forms that the operator may see over the course of the audit 19 Speak clearly and be brief Try to convey your message in a non-adversarial way Do not compare one operator to another Maintain control of the meeting 19

Steps in Conducting a Physical Inspection CSI, Aviation Audit Course Steps in Conducting a Physical Inspection Review standards Obtain checklists Review and “test” Does it meet standard Does not Does Prepare the paper work Proceed with next task Write the audit finding(s) 20 Review Standards ......how can we determine the level of compliance if we don’t know what the standard is...regulation/standard/procedure... by going through your checklist Now, when I know what to measure it against I can move on with the actual physical audit...next I inspect, interview, and test whether it meets the standard… ….if it does not…“Draw up the audit finding”. ….if it does… then proceed to the next 20

Checklists Use as directed Ensure that no area is overlooked CSI, Aviation Audit Course Checklists Use as directed Ensure that no area is overlooked Write down information Check company against CAA regulation requirements 21 One way of knowing the standards or the regulations is by using your checklist…..read ….Your checklist is your best guide to a professional audit ….Best keep it as informal as possible ….Not an inquisition…the goal is to obtain information… 21

Physical Inspections CSI, Aviation Audit Course 22 22 Is the actual inspection of whether the operator is conforming with regulations and its stated policies and standards. An operator policy shall never be more lax than the regulation but may always be stricter than it. When the regulation is lax therefore the auditor holds the operator to its policy instead of the regulation. Physical Inspections 22

Interviewing People Why? CSI, Aviation Audit Course Interviewing People Why? To determine if the system documented in the approved manuals is the system in use To assess the knowledge of the operator’s personnel. To review their duties and responsibilities To determine if any other areas need be audited To tell people what is going on 23 Usual means of communication between department management positions Signs that the channels of communications are effective and positive What are the actions done by management in case of safety issues in reports? Either with pax or with cabin crew 23

Base Inspections Cabin Crew Manuals held by other departments CSI, Aviation Audit Course Base Inspections Cabin Crew Manuals held by other departments and other key Management personnel Manual Amendment & Safety Bulletin distribution, Are CC reports on safety issues dealt with CC injury reports to determine safety related trends Journey logs are closed and repairs corrected Flt Time & Duty Limitations & Rest Requirements Aircraft types Cabin Crew scheduled for & operating 24 CCSM held by other departments I have found are not updated, and the responsibility is not designated by company…shld be in manual I have found that the CML is unanswered or the snags become redundant bec maintenance says they have checked them but when checked it is still inoperative…. Pathlights, smoke detector, reclining mechanisms Check schedules and flight time where crew bases their productive pay Monitoring by managements of Flt time limitations Example is Management or trainer practice of being scheduled to fly after training in the morning because after all they will rest in the outlaying station and it is to their advantage bec they want to go abroad 24

Aircraft Inspection Equipments Exits and Doors Placards CSI, Aviation Audit Course Aircraft Inspection Equipments Exits and Doors Placards Unserviceable equipments 25 Aircraft markings and placards in accordance with the Aircraft Flight Manual often I notice that the instructions for opening doors are not anymore red or missing totally 25

Pre-flight & Inflight Inspections CSI, Aviation Audit Course Pre-flight & Inflight Inspections General No of Cabin Crew & Crew briefing Apron Safety Cabin checks Flt Time & Duty Limitations & Rest Requirements 26 Pre-flight & Flight Inspections Competency cards/licences (presence & validity) trainers forget to log competency checks, a captain has signed in their card Manual inspection (currency & availability) random check (inaccessible to cabin crew Crew briefing are they done properly Correct number of crew (incapacitated crew) The system for reporting unserviceable items or cabin snags and the rectification system.(unanswered by maintenance, open for ages Apron Safety Passenger safety (small airlines complacent of escorting passengers to aircraft) Refueling proc (specially abled pax are told to use WC at this time, cc are busy with loading of aircraft catering and cleaners instead of manning their doors) Cabin Checks Passenger briefing (safety demo, exit row, special needs) (taken for granted by cabin crew, incomplete briefing, inaudible) Carry on baggage procedure/control (size, number, weight, stowage)(plastic bags, portfolios, handbags, cameras, religious images and statues) Alcohol /Drugs – boarding, procedures by ground personnel Flt Time & Duty Limitations & Rest Requirements through interviews Check schedule of cabin crew & and actual take offs and arrivals Actual rest given 26

Cabin Crew Training Program CSI, Aviation Audit Course Cabin Crew Training Program Training Program Crew Qualifications Training Records Dangerous Goods/Hazmat Program CRM 27 Facility – they have moved Program Shld include syllabus/curriculum (for trainers and raters) (as part of the subjects taken should be a go over of the program itself especially for trainers) includes also joint F/D & CC trng what about qualifications for upgrades (who qualifies)(Senior Cabin crew) passing standards and what operator does in case of failures Trainers vis-a vis cabin crew number Qualifications of crew and trainers Records & record system (is it easy to understand, do trainers maintain records properly, including competency cards, retention) Aids (courseware, handout, exams, equipments for training) Monitoring system for training & competency checks Mock ups, Emergency evacuation trainers used doors, tailcone etc. Adequate portable emergency equipment available for training purposes Ensure instructors are qualified for both DG and CRM Ensure CRM is done as per regulation (F/D are normally missing in action, in fact this should be a team including dispathers, maintenance, etc….ground personnel) Workshop type 27

After Interview & Inspection CSI, Aviation Audit Course After Interview & Inspection Make extra notes as necessary Tell Team Leader if necessary 28 ….following the interview and physical inspection, we may want to support any finding of non-conformance with some evidence what is evidence? …………….next slide 28

CSI, Aviation Audit Course Evidence What is it? Anything which supports or shows a non-conformance. 29 (photocopies of training records, photographs, reports, schedules, etc) we are often asked, “what happens if the non-conformance is an immediate threat?” next... 29

Immediate Threat to Safety CSI, Aviation Audit Course Immediate Threat to Safety Inform Team Leader Immediately 30 An immediate threat would be unruly pax, or intoxicated passenger being boarded. Auditor – suppose to observe what crew does abt it… If crew does nothing… say “maybe you shld tell Capt abt it…. Watch again 30

Not an Immediate Threat to safety CSI, Aviation Audit Course Not an Immediate Threat to safety What do you do? 31 when its not an immediate threat (99.9%) what do we do? ...next give it to the operator Confirmation Request Form (CRF)....verbal...they will usually take action If they fix the problem immediately before finishing the audit then that’s the end of it… But if they take longer then it becomes part of the audit finding form 31

CSI, Aviation Audit Course Post Audit 32 Now we go to Post audit 32

Audit Finding Definition CSI, Aviation Audit Course Audit Finding Definition "audit finding" means the determination of non-conformance of a product, process, practice or procedure or a characteristic thereof to a regulation or standard. This will be documented in the Audit Finding Form. 33 Why is the AFF so important? Audit is only as good as the audit report and the report (executive and functional summaries) is supported by the findings. If findings are written incorrectly they can be contested...and thrown out...this could jeopardize the integrity of the entire report. 33

Company De-brief “Exit Meeting" with the operator’s team CSI, Aviation Audit Course Company De-brief “Exit Meeting" with the operator’s team should provide an overview of the audit. 34 The meeting should summarize the audit findings, stating the strength and weakness. Controversial discussion w/ company representatives regarding audit report content must be avoided. 34

Writing Audit Findings CSI, Aviation Audit Course Writing Audit Findings Write the regulatory requirement in terms of chapter, section, paragraph etc. Then incorporate the phrase "Which states" Then write the correct portion of the requirement. The standard and the example(s) must match The example(s) would support more than one Audit Finding 35 35

Writing Examples Ensure that the example matches the regulatory CSI, Aviation Audit Course Writing Examples Ensure that the example matches the regulatory requirement Include a lead-in statement such as "The following are examples of..." where possible Write a statement of how the situation was (past tense) Make reference to supporting evidence 36 reference supporting evidence - “as indicated by the attached journey log pages or photocopies” - “as stated on the attached Confirmation Request Form”, 36

Follow Up CSI, Aviation Audit Course 37 37 Part of follow up is the Corrective Action Plan required 37

Corrective Action Plan CSI, Aviation Audit Course Corrective Action Plan The Operator will then write down their corrective action plan; as CAA auditors we must ensure that corrective action plans are made in such a way that they prevent re-occurrence of the same in the future 38 38

Audit Follow-up The CAA Inspector must continue to CSI, Aviation Audit Course Audit Follow-up The CAA Inspector must continue to monitor the company’s short and long term responses to the audit non conformance findings. The CAA must regulate the company maintaining constant surveillance 39 39

Audit Completed Thank You. CSI, Aviation Audit Course Audit Completed Thank You. 40 40