1. Integra A/S1 WORKSHOP ATM SAFETY REGULATION (OVERSIGHT) January 2006 WORKSHOP ATM SAFETY REGULATION (OVERSIGHT) January 2006

2. Integra A/S2 Welcome and Opening of Workshop Welcome Welcome Objectives of the workshop Objectives of the workshop Programme of the workshop Programme of the workshop

3. Integra A/S3 Objective Introduction to ATM Safety Regulation Introduction to ATM Safety Regulatory Oversight Understanding

4. Integra A/S4 Safety Regulation Objective of Safety regulation: To improve the safety performance of the services provided by the service providers subjected to safety regulation To improve the safety performance of the services provided by the service providers subjected to safety regulation

5. Integra A/S5 Safety Regulation Means of Safety regulation: Standards and requirements Standards and requirements Enforcing adherence to standards Enforcing adherence to standards and requirements and requirements

6. Integra A/S6 Safety Regulation, the Framework Regulatory Functions: Rulemaking & Oversight The States Safety Management Systems The Aviation Activities Collectively: The (national) Safety Management System Management of Safety Safety Management Programme

7. Integra A/S7 Safety Regulation, the Framework Basic Aspects of Safety Management : Geographically Functionally Systematic, pro-active & explicit Uniform & consistent Global ICAO SARPS Independent Nat. regulations

8. Integra A/S8 Safety Regulation, Functions Safety Regulation includes: Rulemaking Safety Regulatory Oversight

9. Integra A/S9 ATM Safety Regulation ATM Safety policy: ATM Safety policy: The National Civil Aviation Administration has safety as its highest priority for the regulation of civil aviation and in the provision of air priority for the regulation of civil aviation and in the provision of air traffic management services. The safety regulation of air traffic management service providers for airspace and aerodromes is a vital function for achieving and maintaining high levels of safety in the delivery of air traffic management services. The regulation will, as an ongoing safety oversight of service providers and services, be carried out impartially, independent of other interests and in a fair and transparent manner, based on published ATM Safety Regulatory requirements.

10. Integra A/S10 ATM Safety Regulation, Functions ATM Safety Regulation includes: ATM Rulemaking ATM Safety Regulatory Oversight

11. Integra A/S11 Safety Regulation, ext. arrangement Safety Regulation shall be (at least functionally) separated from Service Provision Presently, there is a mix of roles, authorities and responsibilities between the DGCA and the AAI During the first visit it was mentioned that a CAA may be form by joining the DGCA and parts of AAI - What is the progress on this issue?

12. Integra A/S12 Safety Regulation, int. arrangement Safety Rulemaking shall be functionally separated from Safety Regulatory Oversight Independence within the regulator

13. Integra A/S13 Safety Regulation, int. arrangement Section B Safety Oversight Section C Analysis/ Statistics Section A Standards & Rule-making Head Organisational arrangement of ATM Safety Regulator:

14. Integra A/S14 ATM Safety Regulation ATM Safety Regulation, Rulemaking Definition: Rulemaking is the process by which safety objectives, requirements and standards are set. It consists of the definition of applicable safety standards, requirements and practices to ensure that both the National ATM Acceptable Level of Safety and regulatory provisions that result from international commitments are observed

15. Integra A/S15 ATM Safety Regulation ATM Safety Regulation, Rulemaking Minimum rulemaking includes: Specification of National ATM Acceptable Level of Safety for the provision of ATS within the airspace Rules and requirements, for the implementation and operation of ATM services: Occurrence reporting Safety Management Systems Risk assessment and mitigation Licensing/certification of personnel

16. Integra A/S16 ATM Safety Regulation ATM Safety Regulatory Oversight Definition: The function undertaken by a designated authority to verify that safety regulatory objectives, requirements and standards are effectively met by the ATM service-providers

17. Integra A/S17 ATM Safety Regulation ATM Safety Regulatory Oversight Oversight embraces: The ATM service-providers Documentation Organisational structure Processes and procedures, including Safety Occurrence Reporting and Safety Management Systems Technical systems Personnel

18. Integra A/S18 ATM Safety Regulation ATM Safety policy: ATM Safety policy: The National Civil Aviation Administration has safety as its highest priority for the regulation of civil aviation and in the provision of air priority for the regulation of civil aviation and in the provision of air traffic management services. The safety regulation of air traffic management service providers for airspace and aerodromes is a vital function for achieving and maintaining high levels of safety in the delivery of air traffic management services. The regulation will, as an ongoing safety oversight of service providers and services, be carried out impartially, independent of other interests and in a fair and transparent manner, based on published ATM Safety Regulatory requirements.

19. Integra A/S19 ATM Safety Regulatory Oversight Objective of ATM Safety Regulatory Oversight To monitor the provision of ATM services in respect of safety (safety performance) and to verify that objectives, requirements, directives and other safety regulation by the ATM Safety Regulator are complied with by the ATM service-providers

20. Integra A/S20 ATM Safety Regulatory Oversight Principles of Safety Regulatory Oversight Support the service providers adherence to the safety standards and requirements Systematic Consistent and uniform Clearly defined terms of reference Factual, objective, unbiased and transparent Performed in co-operation with the service provider Adapted to the situation Outcome understandable and useful to the service provider

21. Integra A/S21 ATM Safety Regulatory Oversight Scope of ATM Safety Regulatory Oversight Regulatory issues Operational and technical issues Licensing and training issues

22. Integra A/S22 ATM Safety Regulatory Oversight Scope of ATM Safety Regulatory Oversight (cont.) Regulatory issues: - Operations manuals, ATS instructions etc. complete, concise and up-to-date - Separation minima - Overview of manoeuvring areas - Low visibility aerodrome operations - Traffic volume/controller work load - Failures/degradation of systems - Reporting procedures - Route structures

23. Integra A/S23 ATM Safety Regulatory Oversight Scope of ATM Safety Regulatory Oversight (cont.) Operational and technical issues: - Working conditions - Automation systems - Equipment - CNS and other safety significant systems - Records

24. Integra A/S24 ATM Safety Regulation Scope of ATM Safety Regulatory Oversight (cont.) Licensing and training issues: - Training and licensing status for ATCOs & other staff - Refresher training - Teamwork competency - English language skills - Standard phraseology

25. Integra A/S25 ATM Safety Regulation Objective of ATM Safety Regulatory Oversight To monitor the provision of ATM services in respect of safety (safety performance) and to verify that objectives, requirements, directives and other safety regulation by the ATM Safety Regulator are complied with by the ATM service-providers

26. Integra A/S26 ATM Safety Regulation Means of ATM Safety Regulatory Oversight Monitoring the safety performance of the service provider Safety regulatory audits to obtain objective evidence as regards the compliance with specific requirements by the service provider

27. Integra A/S27 ATM Safety Regulation Monitoring the safety performance Oversight and review of occurrence reporting systems as well as use of the contents of the reports and the outcome of safety regulatory audits

28. Integra A/S28 ATM Safety Regulation Verifying the compliance with objectives, standards and requirements A process to ensure that ATM service-providers continuously comply with the standards and safety regulatory requirements issued by the ASR Objectives, requirements, standards and other safety-related conditions identified in safety assessment and mitigation documentation to allow the proposed operation of new ATM systems, including transition into operational use, or proposed changes to the operation of existing ATM systems in the form of new developments or modifications are implemented ATM service-providers implement safety-related conditions contained in Safety Directives

29. Integra A/S29 ATM Safety Regulation Linkage between monitoring safety performance and verification of compliance The monitoring process should feed information into the planning of safety regulatory audits identifying areas of concern

30. Integra A/S30 ATM Safety Regulation Initial and on-going safety regulatory oversight Supports the acceptance of A new organisation and significant changes to the organisation of an ATM service-provider Initial oversight New systems and changes to systems

31. Integra A/S31 ATM Safety Regulation Initial and on-going safety regulatory oversight Concerns the continued compliance in respect of Continuous operation of Safety Management Systems and other activities based on safety requirements On-going oversight Continuous operation of in-service ATM systems

32. Integra A/S32 ATM Safety Regulation The cycle of of safety regulatory oversight & relation between regulator and service provider

33. Integra A/S33 ATM Safety Regulation Safety Regulatory Audit An independent and systematic examination conducted by the ATM Safety Regulator to determine whether ATM services or specific parts of an ATM system comply with national requirements and ICAO SARPS Definition

34. Integra A/S34 ATM Safety Regulation Safety Regulatory Audits Apply to, but are not limited to, complete arrangements or elements thereof, to processes, to products or to services Verify compliance of: Written procedures and other established arrangements against required procedures and other required arrangements, and/or Actual processes and their results against written procedures and other established arrangements Scope

35. Integra A/S35 ATM Safety Regulation Safety Regulatory Audits Provide the ASR with evidence of compliance with applicable safety regulatory objectives, requirements and standards and related arrangements by evaluating the need for improvement or corrective action Verify by testing and/or examination whether prescriptive specifications required and previously published by the ASR have been met effectively Be undertaken under the managerial responsibility and overall control of the NCAA, independent of the internal auditing activities undertaken by ATM service-providers as part of their safety management arrangements Shall

36. Integra A/S36 ATM Safety Regulation Safety Regulatory Audits Provide the auditee with an opportunity to correct non- conformities and improve the safety of the area under consideration Shall, cont.

37. Integra A/S37 ATM Safety Regulation Safety Regulatory Audit The physical act of auditing Reaction on any systems weaknesses that have been revealed by the audit A process of:

38. Integra A/S38 ATM Safety Regulation Safety Regulatory Audit Established arrangements against required arrangements Implemented arrangements and their results versus established arrangements and their expected results Focus of the process:

39. Integra A/S39 ATM Safety Regulation Safety Regulatory Audit A process: Required procedures & Required arrangements AUDITING THE PROCESS (which may also involve auditing of product/service) Actual processes & Actual arrangements and their results Written procedures & Written arrangements and their expected results VERIFIED AGAINST Actual parameters of ATM services or parts of the ATM System Required specifications for ATM services or parts of the ATM Systems AUDITING THE PRODUCT/SERVICE APPLICABLE SAFETY STANDARDS & REQUIREMENTS Requirements for processes and other providers arrangements Requirements defining specifications for products or services VERIFIED AGAINST VERIFIED AGAINST Written procedures & Written arrangements

40. Integra A/S40 ATM Safety Regulation Programme of Safety Regulatory Audits The ASR shall establish an annual programme of safety regulatory audits that addresses each and all provider organisations providing ATM services in Thai airspace, and targets all the different ATM services. The programme shall ensure that all entities subject to safety regulatory oversight are audited at least once every xx years.

41. Integra A/S41 ATM Safety Regulation Safety Regulatory Audit Planning Conducting Reporting Corrective action and close-out Four phases:

42. Integra A/S42 ATM Safety Regulation Safety Regulatory Audit Appointing the Audit Team: Audit Team Leader & Auditors Preliminary preparation: - Understanding the subject of auditing - Specific requirements to be verified, the samples - Comm. with auditee, determine visit schedule - Determine the plan of activities The Team Leader communicates with auditee and determines the visit schedule The Auditors prepare the plan of activities of the visit(s) Planning audit:

43. Integra A/S43 ATM Safety Regulation Safety Regulatory Audit Determine the audit basis - Study of auditee (documentation, organisation) and requirements, make High Level Check Lists of specific requirements; and Low Level Check Lists of specific evidence and questions. The lists form basis of plan of activities Planning audit (cont.): Stand. & Req. Hi lev. check lists Low lev. Check lists

44. Integra A/S44 ATM Safety Regulation Safety Regulatory Audit Audit entry meeting, chaired by Audit Team Leader; typical agenda: Conducting audit: Introductions Team Leader and individual audit team members Service provider representatives Purpose / Objectives of the audit (e.g. routine oversight / compliance with requirements, etc.) Scope of the audit (areas of service provider to be audited) Audit visit schedule (times when each area / person will be visited and by whom)

45. Integra A/S45 ATM Safety Regulation Safety Regulatory Audit Audit entry meeting, typical agenda (cont.) Conducting (audit cont.): Audit limitations (audit conclusions based on limited sample / snapshot in time) General administrative arrangements Office facilities etc. Audit guides How results will be formally communicated (reporting mechanism / documentation) Exit meeting Confidentiality Questions

46. Integra A/S46 ATM Safety Regulation Safety Regulatory Audit Audit investigation Conducting audit (cont.): - Interviews - Checks of documentation/records procedures knowledge Use of prepared checklists, but ready to make controlled changes if required. - Careful, consistent recording - Team coordination – team meetings - Audit Tam Leader always in control

47. Integra A/S47 ATM Safety Regulation Safety Regulatory Audit Highlights of auditing techniques: Conducting audit (cont.): - Auditors always need Objective Evidence to verify that a process functions effectively - Objective Evidence should be obtained by auditors examining documents / records etc. of their choosing and by selecting appropriate representative samples. - Auditors should not treat the spoken word of the auditees as objective evidence. This must be obtained by observation.

48. Integra A/S48 ATM Safety Regulation Safety Regulatory Audit Highlights of auditing techniques (cont.): Conducting audit (cont.): - However, information obtained from the spoken word of several individuals may be used as confirmation of understanding by an auditor. - When a non-conformity is found, auditors should take larger samples, sometimes in different parts of the organization, in order to establish if the non-conformity is an isolated incident or a common problem across the organisation

49. Integra A/S49 ATM Safety Regulation Safety Regulatory Audit Audit exit meeting, chaired by Audit Team Leader; typical agenda: Conducting audit: Introductions Team Leader and individual audit team members Service provider representatives Purpose of the meeting Purpose / Objectives of the audit (routine oversight / compliance with ESARRs etc.) Scope of the audit (areas of service provider audited)

50. Integra A/S50 ATM Safety Regulation Safety Regulatory Audit Audit exit meeting, chaired by Audit Team Leader; typical agenda (cont.): Conducting audit: Appreciation of co-operation / assistance, etc. Confidentiality (indicating that all information will remain confidential) Formal report (when it will be sent to the service provider) Audit limitations (conclusions based on limited sample / snapshot in time)

51. Integra A/S51 ATM Safety Regulation Safety Regulatory Audit Audit exit meeting, chaired by Audit Team Leader; typical agenda, (cont.): Conducting audit: Summary statement (main areas of concern revealed by the audit) Audit Findings (non-conformities presented and explained) Regulatory process for corrective action, follow up and audit close out Questions

52. Integra A/S52 ATM Safety Regulation Safety Regulatory Audit Reporting to ATM Safety Regulator Reporting to CAA Reporting to auditee Reporting audit findings:

53. Integra A/S53 ATM Safety Regulation Safety Regulatory Audit Reporting to ATM Safety Regulator (ASR) A report, to include the following to be forwarded to the ASR: date of oversight visit subject of audit auditor(s) observers / specialists accompanying the auditors objectives and scope of the audit summary statement / audit conclusions audit schedule (areas of the service provider visited together with times spent in each area) Reporting audit findings (cont.):

54. Integra A/S54 ATM Safety Regulation Safety Regulatory Audit Reporting to ATM Safety Regulator (ASR) (cont.) overall status of this oversight visit in relation to the NSA annual programme of audits in relation to the organisation being audited details of the specific management system elements / paragraphs sampled status of previously agreed corrective actions (if forming part of this audit) reference documentation used to plan the audit specific documentation/records reviewed during the audit key staff interviewed Reporting audit findings (cont.):

55. Integra A/S55 ATM Safety Regulation Safety Regulatory Audit Reporting to ATM Safety Regulator (ASR) (cont.) specific activities observed details of identified non-compliances supporting details in relation to identified non-compliances general audit observations recommendations to the ASR by the auditor(s) Reporting audit findings (cont.):

56. Integra A/S56 ATM Safety Regulation Safety Regulatory Audit Reporting to CAA A report, to include the audit report and if applicable recommendations is to be forwarded to CAA Reporting audit findings (cont.):

57. Integra A/S57 ATM Safety Regulation Safety Regulatory Audit Reporting to auditee CAA (TBD) will – if possible not later than 2 weeks after the audit, transmit a report to the auditee including the following information Date of oversight visit Auditee Auditor(s) Observers / specialists accompanying the auditors Objectives and scope of the oversight audit Audit schedule (areas of the service provider visited) Details of non-compliances identified by the audit team (including perceived significance) Reporting audit findings (cont.):

58. Integra A/S58 ATM Safety Regulation Safety Regulatory Audit Reporting to auditee (cont.) Response of the ATM service provider to identified non-compliances, Requirements for corrective actions, (including timeframes - determined by perceived significance/impact on safety) Considerations for investigations (relating to auditor(s) general observations) Reporting audit findings (cont.):

59. Integra A/S59 ATM Safety Regulation Safety Regulatory Audit Complete set of records of all audits must be stored with easy access to the records Records:

60. Integra A/S60 ATM Safety Regulation Safety Regulatory Audit Definition Action to eliminate the cause of a detected nonconformity or other undesirable situation Corrective action: NOTE: Corrective action does not mean the action taken to restore a nonconforming situation to a conforming situation. This is known as remedial action. If the root cause of a nonconformity is not addressed then it is very likely that similar nonconformities will recur.

61. Integra A/S61 ATM Safety Regulation Safety Regulatory Audit Responsibility The service provider is responsible taking corrective action Corrective action:

62. Integra A/S62 ATM Safety Regulation Safety Regulatory Audit Process - Regulator finds nonconformity, and informs auditee - Auditee decides upon and takes corrective action - Regulator may perform audit follow-up - Close-out after root cause of nonconformity is eradicated and required safety level is established Corrective action, audit follow-up & close-out:

63. Integra A/S63 ATM Safety Regulation Safety Regulatory Audit Audit management includes - Appointing and organising resources for audits - Elaborating and maintaining audit protocol - Elaborating templates for audit activities - Control of records Audit management:

64. Integra A/S64 ATM Safety Regulation Safety Regulatory Audit Audit management includes - Elaborating and maintaining audit programmes - Monitoring audit activities, their outcome and the feed- back to improve audit effectiveness - Internal audit of audit activities - Ensuring qualifications of auditors Audit management (cont.):

65. Integra A/S65 ATM Safety Regulation Discussion and Conclusion