Normal Operations Safety Survey (NOSS) Chris Henry The University of Texas Human Factors Research Project The University of Texas at Austin 2nd ICAO TEM & NOSS Symposium Washington DC February 7, 2007 UT Human Factors Research Project © Copyright

Presentation Objectives Why conduct NOSS? Factors that lead to a successful NOSS Frequently asked questions UT Human Factors Research Project © Copyright

Aviation Safety Envelope Accidents Incidents Normal Operations Safety UT Human Factors Research Project © Copyright

Safety Data Coverage Accidents Incidents Normal Operations Voluntary Incident Reports NOSS On any given day airlines may collect information from This diagram represents sources of aviation safety data We collect 4 main types of informaiton with the safety envelope: Accident investigations – rarest, non-voluntary, reactive investigations FOQA – technical flight parameters ASAP – self-report data resulting from voluntary crew reporting, proactive, center on FAR violations LOSA – normal line operations, info from SME’s on normal flights At UT – we have developed a multi-project research program that covers assessment and analysis of 3 of these data collection programs - reaching each level of the safety envelope Accident Investigation Mandatory Incident Reports UT Human Factors Research Project © Copyright

Aviation Safety Envelope Accidents Incidents Normal Operations NOSS Rationale Proactive snapshot of system / controller performance strengths and weaknesses in normal operations (just like a health checkup) On any given day airlines may collect information from This diagram represents sources of aviation safety data We collect 4 main types of informaiton with the safety envelope: Accident investigations – rarest, non-voluntary, reactive investigations FOQA – technical flight parameters ASAP – self-report data resulting from voluntary crew reporting, proactive, center on FAR violations LOSA – normal line operations, info from SME’s on normal flights At UT – we have developed a multi-project research program that covers assessment and analysis of 3 of these data collection programs - reaching each level of the safety envelope UT Human Factors Research Project © Copyright

What can NOSS provide? Identify threats within the operating environment Check the quality/usability of procedures Identify problems with controller/equipment interfaces Assess the degree of transference of training to the line Understand controller shortcuts and workarounds Make comparisons across units or facilities Assess safety margins UT Human Factors Research Project © Copyright

NOSS Success Factors NOSS success is dependent upon methodology and execution Formal Check Regulator Nobody Angel Performance NOSS Observer Natural Performance - Controller Trust + - + - NOSS value + - + Low controller trust = Low quality data because there will be no differentiation between NOSS and proficiency checks UT Human Factors Research Project © Copyright

NOSS Operating Characteristics Over-the-shoulder observations during normal shifts No observations of controllers who are undergoing training No observations while checks are being conducted Voluntary controller participation Controllers have a right to decline a NOSS observation Denial rates in NOSS trials have been low Joint management / association sponsorship Letter signed by management & association representatives sent to all controllers UT Human Factors Research Project © Copyright

NOSS Operating Characteristics Trusted and trained observers Trust and Credibility Selection process – management / association list of candidates Diversity of observers increases buy-in Training Classroom training, test observations, recalibration session Curriculum TEM principles and exercises NOSS observation protocols Narrative guidelines UT Human Factors Research Project © Copyright

NOSS Operating Characteristics Anonymous, confidential, and non-punitive data collection No names, employee numbers, dates, experience, or other identifying information Data used for safety purposes only – no punitive actions Observers identity is known only by the third party facilitator Trusted and secure data collection site Third party or controller association gatekeeper Controllers and observers must be comfortable with the data collection site Systematic data collection instrument No judgments – observers record data based on TEM events UT Human Factors Research Project © Copyright

NOSS Safety Metrics What do controllers do to manage their airspace? Manage operational complexity 1 Threat Management Manage their own errors 2 Error Management 3 Manage deviations in airspace status Undesired State Management The Threat and Error Management Model serves as the “NOSS Safety Metrics” UT Human Factors Research Project © Copyright

NOSS Operating Characteristics Data verification process TEM data checked to ensure coding accuracy and consistency with SOP Data analysis does not begin until verification has been completed Data-derived targets for safety enhancement Serve as benchmarks for safety change Initial NOSS, safety change process, follow-up NOSS Feedback results to controllers Results summarized for controllers Information on how organization intends to respond to the data UT Human Factors Research Project © Copyright

NOSS: Gaining Controller Trust Over-the-shoulder observations during normal shifts Joint management / association sponsorship Voluntary Participation Trusted and trained observers Anonymous, confidential, and non-punitive data collection Trusted and secure data collection site Systematic data collection instrument Data verification process Data-derived targets for safety enhancement Feedback results to controllers UT Human Factors Research Project © Copyright

NOSS Defined The 10 characteristics that differentiate NOSS (LOSA) from other methodologies have been endorsed by the: Federal Aviation Administration International Civil Aviation Organization International Air Transport Association International Federation of Airline Pilots Association International Federation of Air Traffic Controllers (6) The LOSA Collaborative US Airline Pilots Association University of Texas Programs that omit one or more of the characteristics might be useful and beneficial, but they are not NOSS NOSS must have all ten characteristics UT Human Factors Research Project © Copyright

Frequent Questions How long are NOSS observations? Observations approximately one hour in duration have given us good data Do we observe individual controllers/positions or teams? Team dynamics captured, but from the perspective of an individual operating position Should NOSS be continuous or a snapshot? Snapshot allows organization to assess and act on data Other data sources can inform between snapshots Little benefit for the increased cost of doing continuous NOSS What should the scope of a NOSS be? Depends on the size of the facilities and the comparisons to be made UT Human Factors Research Project © Copyright