Ensuring the Safety of Future Developments Peter Stastny Head of Safety Regulation Unit EUROCONTROL peter.stastny@eurocontrol.int ICAO Montreal, Thursday 29 March 2007

Overview A Performance-based Approach to Safety Monitoring Safety Performance Performance Measurement – the essential tool Measuring Safety Maturity Performance-based Approach to Managing Risk Risk management – part of SMS Safety oversight aspects – the role of ESARR 1 Risk classification methodology – defining tolerable safety Conclusions

“Historic” Safety Performance Measurement Performance – part of SMS Recruitment/ Selection Procedures Operational Processes Risk Assessment New Systems Incident Reporting Safety Surveys and Follow-up Training Risk Assessment ATM Procedures Incident Investigation Interface ATS CNS, AIM, Airports Competency Checks Lessons Learnt CNS /AIM Maintenance Procedures Risk Assessment Airspace Changes Refresher/ Advanced Training “Historic” Safety Performance Measurement Emergency Procedures Risk Assessment Software Changes Safety Maturity Measurement Proactive Reactive A systematic approach to the management of safety

Performance – part of Safety Oversight ESARR 1…. Defines minimum arrangements/ processes for ATM safety oversight: with certification or without certification A unique basis for harmonising and reinforcing the role and operation of national regulatory bodies Requires monitoring of safety performance as part of safety oversight I N T H E F U T U R E . . . ESARR 1 SAFETY OVERSIGHT IN ATM EUROCONTROL SAFETY REGULATORY REQUIREMENT (ESARR) EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EDITION : 1.0 EDITION DATE : 27-02-2002 STATUS : RELEASED ISSUE CLASS : GENERAL PUBLIC

Safety Performance Measurement KPIs Performance Indicators Information Management Measurement Metrics Laws Incidents Accidents Culture Audit Compliance SMS Procedures Resources AIB Recommendations Public/ Industry States/ Interested Parties Organisational Level (Service Providers) Key Principles Information to public/stakeholders Call to action by stakeholders Facilitates identification of scope of action required Facilitates management of improvement of service The whole process needs to be a continuous improvement activity

Occurrence-based performance measurement EUROCONTROL has developed safety data reporting to identify key risk areas at European level…

Management Statement in Establishing a Just Culture Gross negligence Omissions Slips Lapses Mistakes Violations Criminal Offences LAWS unintentional Management Statement in Safety Policy deliberate deliberate Procedures Proactive Management

ATM Safety System Maturity in ECAC States Independent maturity assessment system Applied across ECAC Region Now being expanded to neighbouring States

Performance-based Approach to Managing Risk Recruitment/ Selection Procedures Operational Processes Risk Assessment New Systems Incident Reporting Safety Surveys and Follow-up Training Risk Assessment ATM Procedures Incident Investigation Interface ATS CNS, AIM, Airports Competency Checks Lessons Learnt CNS /AIM Maintenance Procedures Risk Assessment Airspace Changes Refresher/ Advanced Training Emergency Procedures Risk Assessment Software Changes Risk Assessment and Mitigation Proactive Reactive A systematic approach to the management of safety

Risk Assessment and Mitigation - 1 Empirical methods of risk assessment no longer sufficient Systems more complex – failure modes more difficult to identify Mitigation methods are more complex too – and more costly Performance-based approach to mitigation is needed – what are the design targets to be met?

Risk Assessment and Mitigation - 2 Transparency is also required by those who will: Own and operate the system Ultimately rely on the safety of the system Bear liability if the system fails A formal, structured and visible approach is the only answer It is required by ESARR 4 and the EC’s Common Requirements for ANS provision A risk classification scheme is a necessary start point for the decision-making that must follow

This is the approach being implemented in Risk Management Risk Management is primarily… …a task for the service provider The provider / operator manages the system and its hazards Risk management processes are conducted as part of a Safety Management System Legal requirement for service providers to conduct risk assessment and mitigation in relation to the implementation of changes to the ATM system This is the approach being implemented in Air Traffic Management in Europe

RISK ASSESSMENT AND MITIGATION ACTIVITIES Risk Management This is the sort of process required in ESARR 4 ... DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION PROJECT RISK ASSESSMENT AND MITIGATION ACTIVITIES ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables ACCEPTANCE REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION

Risk Management This has to be done by the provider... PROJECT PROJECT DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION PROJECT ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables RISK ASSESSMENT AND MITIGATION ACTIVITIES ACCEPTANCE REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION PROJECT ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables RISK ASSESSMENT AND MITIGATION ACTIVITIES ACCEPTANCE REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION

WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ? Risk Management But what about this ? DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION PROJECT ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables RISK ASSESSMENT AND MITIGATION ACTIVITIES ACCEPTANCE REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION DETERMINATION & SPECIFICATION DESIGN AND DEVELOPMENT INSTALLATION AND TRANSITION OPERATION PROJECT ‘SAFETY CASE’ Risk Assessment and Mitigation Deliverables RISK ASSESSMENT AND MITIGATION ACTIVITIES ACCEPTANCE REVIEW OF THE RISK ASSESSMENT AND MITIGATION DOCUMENTATION WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?

Acceptance of new systems and changes EUROCONTROL Acceptance of new systems and changes The provider… The regulator… In some cases, the provider decides about the change… … using risk assessment and mitigation process to support its internal decision-making. This is possible if: The provider’s process is demonstrated to be effective, Enough safety oversight is focused on these processes (e.g. by means of audits) Regulators may identify new systems and changes… … to be directly accepted by the regulatory authority through a formal acceptance (or approval) The Regulator makes the final decision on the acceptability of the system to go into operation The review of the ‘safety case’ provides the Regulator with evidence to support his decision

WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ? EUROCONTROL WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ? The ESARR 1 Approach The ESARR 1 process for the safety oversight of changes to the ATM system: Is implemented by the Regulator by considering results from the risk assessment and mitigation process conducted by the provider Defines a minimum category of changes, whose safety case must be reviewed by the Regulator… …Based on the severity of the hazards identified by the provider in relation to the change Provides the regulator with discretion to review other changes

EUROCONTROL Yellow = provider Red = regulator WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ? The Role Of Oversight Provider conducts risk assessment and mitigation and produces a ‘safety case’ Planned Change (new system or change to existing system) Implementation by the provider of the change (as accepted by the regulator) REGULATOR REVIEWS SAFETY CASE REGULATOR APPLIES DIFFERENT APPROACH DEPENDING ON THE CHANGE Acceptance by the regulator Additional Safety Conditions imposed Major Minor Accepted through ATM provider’s procedures (which are subject to regulator’s auditing) REGULATOR CONDUCTS SAFETY REGULATORY AUDITS MAJOR = Those changes whose assessment of the potential effects of hazards on the safety of aircraft, conducted by the provider in accordance with ESARR 4, identifies hazards with potential to lead to an accident or serious incident Other changes that the Regulator considers appropriate to review

Summarising the Approach to Risk Management European ATM service providers are required to implement risk assessment and mitigation as part of their SMS: Risk assessment and mitigation processes are subject to regulatory auditing as any other safety-related process In addition, the Regulator will specifically review the results of these processes in relation to, at least, the most critical safety-related changes The implementation of these changes will be subject to regulatory acceptance based on the results.

Risk Classification Scheme We now have a severity classification scheme for the identification of the effects of ATM/CNS related hazards on the safety of aircraft. (EC law) We also have a risk classification scheme with a maximum tolerable probability for ATM directly contributing to accidents in the ECAC region (severity class 1) ….but maximum tolerable probability for the severity classes 2 to 5 have still to be developed. States, EC and EUROCONTROL acting together to complete and update those probabilities, Development of regulatory material for the establishment of a quantified risk classification scheme at regulatory level .

Identifying Tolerability of Change Hazard identification. Safety target effects Severity likelihood Severity of the effect Catastrophic Major Average Minor No effect 1 2 3 4 5 I II Likelihood III IV V VI Risk Mitigation Tolerable? no yes Safety objectives Continue the design

Conclusions - 1 Performance-based ATM framework… We are on the way… good progress being made. Experience so far… A performance-driven approach requires: - Data (occurrences, maturity etc.) “Just Culture” – overcoming inhibitors to progress A measurement system, harmonised globally Analysis capability Key Performance Indicators (ultimately)

Conclusions - 2 We’ve had a risk-based approach to the management of safety for decades, but…. The risks are more difficult to identify now Move from “historic” to “predictive” risk assessment A formal, visible assurance methodology We need systems to measure the risks before and after changes to the system (was mitigation successful?) A fully functioning SMS will provide the tools to do the job

Global needs in safety: - Conclusions - 3 Global needs in safety: - A common approach to safety – management and regulation Common minimum levels of safety Availability of information on which to base a performance-driven approach Common safety “language” – terms, taxonomy and appreciation of risk The correct balance between State functions and those of other stakeholders

Ensuring the Safety of Future Developments Peter Stastny Head of Safety Regulation Unit EUROCONTROL peter.stastny@eurocontrol.int ICAO Montreal, Thursday 29 March 2007