1. Ensuring the Safety of Future Developments
Peter Stastny Head of Safety Regulation Unit EUROCONTROL ICAO Montreal, Thursday 29 March 2007

2. Overview A Performance-based Approach to Safety
Monitoring Safety Performance
Performance Measurement – the essential tool
Measuring Safety Maturity
Performance-based Approach to Managing Risk
Risk management – part of SMS
Safety oversight aspects – the role of ESARR 1
Risk classification methodology – defining tolerable safety
Conclusions

3. “Historic” Safety Performance Measurement
Performance – part of SMS
Recruitment/
Selection
Procedures Operational Processes
Risk Assessment New Systems
Incident Reporting
Safety Surveys and Follow-up
Training
Risk Assessment ATM Procedures
Incident Investigation
Interface ATS CNS, AIM, Airports
Competency
Checks
Lessons Learnt
CNS /AIM
Maintenance Procedures
Risk Assessment Airspace Changes
Refresher/
Advanced Training
“Historic” Safety Performance Measurement
Emergency Procedures
Risk Assessment Software Changes
Safety Maturity Measurement
Proactive
Reactive
A systematic approach to the management of safety

4. Performance – part of Safety Oversight
ESARR 1….
Defines minimum arrangements/ processes for ATM safety oversight:
with certification
or without certification
A unique basis for harmonising and reinforcing the role and operation of national regulatory bodies
Requires monitoring of safety performance as part of safety oversight
I N T H E F U T U R E . . .
ESARR 1
SAFETY OVERSIGHT IN ATM
EUROCONTROL SAFETY REGULATORY REQUIREMENT
(ESARR)
EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION
EDITION :
EDITION DATE :
STATUS : RELEASED ISSUE
CLASS : GENERAL PUBLIC

5. Safety Performance Measurement
KPIs
Performance
Indicators
Information
Management
Measurement Metrics
Laws
Incidents
Accidents
Culture
Audit Compliance
SMS Procedures
Resources
AIB Recommendations
Public/
Industry
States/
Interested Parties
Organisational
Level
(Service Providers)
Key Principles
Information to public/stakeholders
Call to action by stakeholders
Facilitates identification of scope of action required
Facilitates management of improvement of service
The whole process needs to be a continuous improvement activity

6. Occurrence-based performance measurement
EUROCONTROL has developed safety data reporting to identify key risk areas at European level…

7. Management Statement in
Establishing a Just Culture
Gross
negligence
Omissions
Slips
Lapses
Mistakes
Violations
Criminal
Offences
LAWS
unintentional
Management Statement in
Safety Policy
deliberate
deliberate
Procedures
Proactive
Management

8. ATM Safety System Maturity in ECAC States
Independent maturity assessment system
Applied across ECAC Region
Now being expanded to neighbouring States

9. Performance-based Approach to Managing Risk
Recruitment/
Selection
Procedures Operational Processes
Risk Assessment New Systems
Incident Reporting
Safety Surveys and Follow-up
Training
Risk Assessment ATM Procedures
Incident Investigation
Interface ATS CNS, AIM, Airports
Competency
Checks
Lessons Learnt
CNS /AIM
Maintenance Procedures
Risk Assessment Airspace Changes
Refresher/
Advanced Training
Emergency Procedures
Risk Assessment Software Changes
Risk Assessment and Mitigation
Proactive
Reactive
A systematic approach to the management of safety

10. Risk Assessment and Mitigation - 1
Empirical methods of risk assessment no longer sufficient
Systems more complex – failure modes more difficult to identify
Mitigation methods are more complex too – and more costly
Performance-based approach to mitigation is needed – what are the design targets to be met?

11. Risk Assessment and Mitigation - 2
Transparency is also required by those who will:
Own and operate the system
Ultimately rely on the safety of the system
Bear liability if the system fails
A formal, structured and visible approach is the only answer
It is required by ESARR 4 and the EC’s Common Requirements for ANS provision
A risk classification scheme is a necessary start point for the decision-making that must follow

12. This is the approach being implemented in
Risk Management
Risk Management is primarily…
…a task for the service provider
The provider / operator manages the system and its hazards
Risk management processes are conducted as part of a Safety Management System
Legal requirement for service providers to conduct risk assessment and mitigation in relation to the implementation of changes to the ATM system
This is the approach being implemented in
Air Traffic Management in Europe

13. RISK ASSESSMENT AND MITIGATION ACTIVITIES
Risk Management
This is the sort of process required in ESARR 4 ...
DETERMINATION
& SPECIFICATION
DESIGN AND
DEVELOPMENT
INSTALLATION
AND TRANSITION
OPERATION
PROJECT
RISK ASSESSMENT AND MITIGATION ACTIVITIES
‘SAFETY CASE’
Risk Assessment
and Mitigation
Deliverables
ACCEPTANCE
REVIEW OF THE RISK ASSESSMENT
AND MITIGATION DOCUMENTATION

14. Risk Management This has to be done by the provider... PROJECT PROJECT
DETERMINATION
& SPECIFICATION
DESIGN AND
DEVELOPMENT
INSTALLATION
AND TRANSITION
OPERATION
PROJECT
‘SAFETY CASE’
Risk Assessment
and Mitigation
Deliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCE
REVIEW OF THE RISK ASSESSMENT
AND MITIGATION DOCUMENTATION
DETERMINATION & SPECIFICATION
DESIGN AND
DEVELOPMENT
INSTALLATION
AND TRANSITION
OPERATION
PROJECT
‘SAFETY CASE’
Risk Assessment
and Mitigation
Deliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCE
REVIEW OF THE RISK ASSESSMENT
AND MITIGATION DOCUMENTATION

15. WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?
Risk Management
But what about this ?
DETERMINATION
& SPECIFICATION
DESIGN AND
DEVELOPMENT
INSTALLATION
AND TRANSITION
OPERATION
PROJECT
‘SAFETY CASE’
Risk Assessment
and Mitigation
Deliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCE
REVIEW OF THE RISK ASSESSMENT
AND MITIGATION DOCUMENTATION
DETERMINATION & SPECIFICATION
DESIGN AND
DEVELOPMENT
INSTALLATION
AND TRANSITION
OPERATION
PROJECT
‘SAFETY CASE’
Risk Assessment
and Mitigation
Deliverables
RISK ASSESSMENT AND MITIGATION ACTIVITIES
ACCEPTANCE
REVIEW OF THE RISK ASSESSMENT
AND MITIGATION DOCUMENTATION
WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?

16. Acceptance of new systems and changes
EUROCONTROL
Acceptance of new systems and changes
The provider…
The regulator…
In some cases, the provider decides about the change…
… using risk assessment and mitigation process to support its internal decision-making.
This is possible if:
The provider’s process is
demonstrated to be effective,
Enough safety oversight is
focused on these processes
(e.g. by means of audits)
Regulators may identify new systems and changes…
… to be directly accepted by the regulatory authority through a formal acceptance (or approval)
The Regulator makes the final decision on the acceptability of the system to go into operation
The review of the ‘safety case’ provides the Regulator with evidence to support his decision

17. WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?
EUROCONTROL
WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?
The ESARR 1 Approach
The ESARR 1 process for the safety oversight of changes to the ATM system:
Is implemented by the Regulator by considering results from the risk assessment and mitigation process conducted by the provider
Defines a minimum category of changes, whose safety case must be reviewed by the Regulator…
…Based on the severity of the hazards identified by the provider in relation to the change
Provides the regulator with discretion to review other changes

18. EUROCONTROL
Yellow = provider
Red = regulator
WHO ACCEPTS THE INTRODUCTION OF NEW SYSTEMS AND CHANGES ?
The Role Of Oversight
Provider conducts risk assessment and mitigation and produces a ‘safety case’
Planned Change
(new system or change to existing system)
Implementation
by the provider of the change (as accepted by the regulator)
REGULATOR REVIEWS SAFETY CASE
REGULATOR APPLIES DIFFERENT APPROACH DEPENDING ON THE CHANGE
Acceptance
by the regulator
Additional Safety
Conditions
imposed
Major
Minor
Accepted through
ATM provider’s procedures
(which are subject to
regulator’s auditing)
REGULATOR
CONDUCTS
SAFETY
REGULATORY
AUDITS
MAJOR =
Those changes whose assessment of the potential effects of hazards on the safety of aircraft, conducted by the provider in accordance with ESARR 4, identifies hazards with potential to lead to an accident or serious incident
Other changes that the Regulator considers appropriate to review

19. Summarising the Approach to Risk Management
European ATM service providers are required to implement risk assessment and mitigation as part of their SMS:
Risk assessment and mitigation processes are subject
to regulatory auditing as any other safety-related process
In addition, the Regulator will specifically review the results of these processes in relation to, at least, the most critical safety-related changes
The implementation of these changes will be subject to regulatory acceptance based on the results.

20. Risk Classification Scheme
We now have a severity classification scheme for the identification of the effects of ATM/CNS related hazards on the safety of aircraft. (EC law)
We also have a risk classification scheme with a maximum tolerable probability for ATM directly contributing to accidents in the ECAC region (severity class 1) ….but
maximum tolerable probability for the severity classes 2 to 5 have still to be developed.
States, EC and EUROCONTROL acting together to complete and update those probabilities,
Development of regulatory material for the establishment of a quantified risk classification scheme at regulatory level .

21. Identifying Tolerability of Change
Hazard
identification.
Safety
target
effects
Severity
likelihood
Severity of the effect
Catastrophic
Major
Average
Minor
No effect 1 2 3 4 5 I II
Likelihood
III IV V VI
Risk
Mitigation
Tolerable? no
yes
Safety objectives
Continue the design

22. Conclusions - 1
Performance-based ATM framework… We are on the way… good progress being made. Experience so far…
A performance-driven approach requires: -
Data (occurrences, maturity etc.)
“Just Culture” – overcoming inhibitors to progress
A measurement system, harmonised globally
Analysis capability
Key Performance Indicators (ultimately)

23. Conclusions - 2
We’ve had a risk-based approach to the management of safety for decades, but….
The risks are more difficult to identify now
Move from “historic” to “predictive” risk assessment
A formal, visible assurance methodology
We need systems to measure the risks before and after changes to the system (was mitigation successful?)
A fully functioning SMS will provide the tools to do the job

24. Global needs in safety: -
Conclusions - 3
Global needs in safety: -
A common approach to safety – management and regulation
Common minimum levels of safety
Availability of information on which to base a performance-driven approach
Common safety “language” – terms, taxonomy and appreciation of risk
The correct balance between State functions and those of other stakeholders

25. Ensuring the Safety of Future Developments
Peter Stastny Head of Safety Regulation Unit EUROCONTROL ICAO Montreal, Thursday 29 March 2007