Presentation is loading. Please wait.

Presentation is loading. Please wait.

The LOSA Archive: The data and how it can be used Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) 2 nd ICAO Global.

Similar presentations


Presentation on theme: "The LOSA Archive: The data and how it can be used Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) 2 nd ICAO Global."— Presentation transcript:

1 The LOSA Archive: The data and how it can be used Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) 2 nd ICAO Global Symposium on Threat and Error Management (TEM) and Normal Operations Safety Survey (NOSS) in Air Traffic Control 7-8 February 2007

2 Purpose of Presentation Show how the LOSA Archive is used in the airline world Show how the LOSA Archive is used in the airline world Show how a similar NOSS Archive could benefit the ATC world Show how a similar NOSS Archive could benefit the ATC world

3 What is LOSA? Line Operations Safety Audits Line Operations Safety Audits Forerunner/Prototype for NOSS Forerunner/Prototype for NOSS Normal operations monitoring based on cockpit observations of regularly scheduled commercial flights Normal operations monitoring based on cockpit observations of regularly scheduled commercial flights Trained expert observers using TEM taxonomy to frame, describe, code the flight Trained expert observers using TEM taxonomy to frame, describe, code the flight ICAO: Industry best practice for normal operations monitoring (ICAO LOSA Manual, Doc 9803) ICAO: Industry best practice for normal operations monitoring (ICAO LOSA Manual, Doc 9803) FAA: (FAA Advisory Circular ) FAA: (FAA Advisory Circular )

4 TEM in the Airline World Threats: Adverse weather, traffic congestion, terrain, airport conditions, aircraft malfunctions, problems with ground, maintenance, dispatch, cabin, and of course….. ATC Threats: Adverse weather, traffic congestion, terrain, airport conditions, aircraft malfunctions, problems with ground, maintenance, dispatch, cabin, and of course….. ATC Errors: Handling, automation, systems, procedures, briefings, communication Errors: Handling, automation, systems, procedures, briefings, communication Undesired Aircraft States: Speed, lateral and vertical deviations, unstable approaches, incorrect engine/system configurations, taxiway incursions Undesired Aircraft States: Speed, lateral and vertical deviations, unstable approaches, incorrect engine/system configurations, taxiway incursions

5 The LOSA Archive (Archie) A database maintained and updated by UT A database maintained and updated by UT Data from airlines who do a LOSA with TLC Data from airlines who do a LOSA with TLC As of February 2007, data from: As of February 2007, data from: 30 airline LOSAs (including 4 repeat LOSAs) 30 airline LOSAs (including 4 repeat LOSAs) ~6000 regularly scheduled commercial flights ~6000 regularly scheduled commercial flights 22,000+ threats 22,000+ threats 15,000+ errors 15,000+ errors 2,500+ undesired aircraft states 2,500+ undesired aircraft states

6 LOSA Airlines

7 Archive: Three Main Uses 1. Benchmarking 2. Industry Trends 3. Interface with other safety information sources

8 Archive Use #1: Benchmarking Empirical TEM data Empirical TEM data New form of data - One hand clapping New form of data - One hand clapping Example: Airline X Example: Airline X 36% of flights had a mismanaged threat 36% of flights had a mismanaged threat 30% of flights had a mismanaged Handling error 30% of flights had a mismanaged Handling error 26% of flights had some form of intentional noncompliance. 26% of flights had some form of intentional noncompliance. Is that good, bad or average? Is that good, bad or average?

9 Airline X and the LOSA Archive 36% of flights had a mismanaged threat 36% of flights had a mismanaged threat Average across 30 LOSAs is 34%. Range 16-63% Average across 30 LOSAs is 34%. Range 16-63% 30% of flights had a mismanaged Handling error 30% of flights had a mismanaged Handling error Average is 37%. Range 13-88% Average is 37%. Range 13-88% 26% of flights had some form of intentional noncompliance. 26% of flights had some form of intentional noncompliance. Average is 46%. Range 22-90% Average is 46%. Range 22-90%

10 Archive Use #2: Industry Trends Normal Ops monitoring provides important base rate information Normal Ops monitoring provides important base rate information Example: Unstable Approaches Example: Unstable Approaches We know how many incidents/accidents involve an unstable approach, but do we know how many unstable approaches occur without incident? We know how many incidents/accidents involve an unstable approach, but do we know how many unstable approaches occur without incident? Average is 5% of flights. Range 0-15% across 30 LOSAs Average is 5% of flights. Range 0-15% across 30 LOSAs 10% of UASs are linked to a mismanaged ATC threat 10% of UASs are linked to a mismanaged ATC threat

11 ATC Threats in the LOSA Archive Data are based on ~2400 Flights (~9500 threats) ATC the most common threat ¼ of all threats were ATC threats 10% of all ATC threats were mismanaged by the crews In all, 30% of all mismanaged threats were ATC

12 ATC threats 2426 Flights 2349 ATC threats Challenging clearances/ late change (46%) Runway Changes (14%) ATC error (13%) Other ATC threats (20%) Language difficulty (7%)

13 Mismanaged ATC threats 2426 Flights 2349 ATC threats 236 mismanaged Challenging clearances (53%) Runway Changes (14%) Language difficulty (8%) Similar call signs (5%) ATC error (7%)

14 ATC threats -> Crew Errors 2426 Flights 236 mismanaged ATC threats 280 errors Automation errors (20%) – wrong MCP/FCU altitude setting dialed Communication errors (27%) - misinterpret ATC instructions Aircraft handling (24%) – unintentional speed, lateral, vertical deviations Cross-verification errors (7%) Ground navigation errors (6%)

15 ATC threats -> Crew Errors -> UAS 2426 Flights 236 mismanaged ATC threats 280 errors 56 Automation errors 76 Communication errors 67 Aircraft handling errors 19 Cross-verification errors 16 Ground navigation errors 2 UAS 58 UAS 5 UAS 9 UAS 21 UAS

16 ATC threats -> UAS 2426 Flights 236 mismanaged ATC threats 280 errors 106 UAS Lateral deviation 17% Speed too high 14% Vertical deviation 10% Unstable Approach 10% Continued Landing 9% Incorrect Automation configuration 9% Taxiway/runway incursion 6% Speed too low 6% Incorrect Aircraft configuration 5% Other 14% Bottom line: 4% of flights had a UAS arising from an ATC threat that was mismanaged. ¾ occurred during descent/approach/land

17 Archive Use #2: Industry Trends What does Noncompliance signify? Has to meet one of three conditions to be coded as intentional noncompliance in LOSA: Committed multiple times during one phase of flight (e.g., missing multiple altitude callouts during descent). (coded as one intentional noncompliance error) Committed multiple times during one phase of flight (e.g., missing multiple altitude callouts during descent). (coded as one intentional noncompliance error) Crew openly discusses they are intentionally committing an action that is against published SOP Crew openly discusses they are intentionally committing an action that is against published SOP Observer determines the crew is time-optimizing SOP when time is otherwise available (i.e., performing a checklist from memory) Observer determines the crew is time-optimizing SOP when time is otherwise available (i.e., performing a checklist from memory) Most errors are coded as unintentional/ honest mistakes. If observer is in doubt, error is coded as unintentional.

18 Is Noncompliance a Problem? The average (across 30 LOSAs): 46% of flights have one or more noncompliance errors The average (across 30 LOSAs): 46% of flights have one or more noncompliance errors Range: 22% of flights at one airline to 90% at another Range: 22% of flights at one airline to 90% at another Most common noncompliance errors Most common noncompliance errors checklist performed from memory / nonstandard checklist use checklist performed from memory / nonstandard checklist use failure to cross-verify MCP/FCU altitude changes failure to cross-verify MCP/FCU altitude changes PF makes own MCP/FCU changes when hand-flying PF makes own MCP/FCU changes when hand-flying Conclusion 1: Noncompliance is just experienced pilots taking optimizing shortcuts… no big deal Conclusion 1: Noncompliance is just experienced pilots taking optimizing shortcuts… no big deal

19 Industry Trend: What does Noncompliance signify? Across 30 LOSAs, airlines that have higher rates of intentional noncompliance also have higher rates of: Mismanaged threats (r =.60) Mismanaged threats (r =.60) Mismanaged errors - handling (r =.81) Mismanaged errors - handling (r =.81) Mismanaged errors - procedural (r =.80) Mismanaged errors - procedural (r =.80) Undesired aircraft states UAS (r =.81) Undesired aircraft states UAS (r =.81) Mismanaged UASs (r =.73) Mismanaged UASs (r =.73) Conclusion 2: Increased noncompliance decreases safety margins… Noncompliance reflects the safety culture… Conclusion 2: Increased noncompliance decreases safety margins… Noncompliance reflects the safety culture…

20 Archive Use #3: Interface with other safety information sources Queries from / data- sharing with: Airlines Airlines Incident Reporting systems Incident Reporting systems IATA/ICAO (ITA) IATA/ICAO (ITA) Transport Safety Boards Transport Safety Boards Boeing Boeing ATC (NOSS Archive?) ATC (NOSS Archive?)

21 And so - a NOSS Archive? Have to be patient while the methodology matures and the Archive grows, but once in place: You can benchmark within and across facilities to determine strengths and vulnerabilities You can benchmark within and across facilities to determine strengths and vulnerabilities You can trend to determine industry best practices & problems You can trend to determine industry best practices & problems A TEM-based NOSS Archive would allow ATC to converse freely with the LOSA Archive on matters of mutual interest to pilots and controllers A TEM-based NOSS Archive would allow ATC to converse freely with the LOSA Archive on matters of mutual interest to pilots and controllers

22 The University of Texas Human Factors Research Project


Download ppt "The LOSA Archive: The data and how it can be used Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) 2 nd ICAO Global."

Similar presentations


Ads by Google