Presentation is loading. Please wait.

Presentation is loading. Please wait.

A Glimpse at Archie: The LOSA Archive Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) First ICAO Global Symposium on.

Similar presentations


Presentation on theme: "A Glimpse at Archie: The LOSA Archive Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) First ICAO Global Symposium on."— Presentation transcript:

1 A Glimpse at Archie: The LOSA Archive Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) First ICAO Global Symposium on TEM & NOSS in ATC 9-10 November, 2005, Luxembourg

2 What is Archie? Archie is the LOSA Archive Archie is the LOSA Archive A database of de-identified airline data which is maintained and updated by UT A database of de-identified airline data which is maintained and updated by UT Airlines that do a LOSA with The LOSA Collaborative contribute their data to Archie for research purposes Airlines that do a LOSA with The LOSA Collaborative contribute their data to Archie for research purposes Archie provides the airline perspective Archie provides the airline perspective

3 How old is Archie? Five LOSAs in five years to develop the methodology Five LOSAs in five years to develop the methodology Data in the Archive starts in 2000 Data in the Archive starts in 2000 Had to wait for Archie to grow before doing any benchmarking or industry-wide analyses. Had to wait for Archie to grow before doing any benchmarking or industry-wide analyses. If we think of Archie as a dog, he would be a young bloodhound If we think of Archie as a dog, he would be a young bloodhound

4 How big is Archie? As of September 2005, Archie has data from As of September 2005, Archie has data from 23 airline LOSAs 23 airline LOSAs 4800 flights 4800 flights 4800 flight narratives written by trained observers 4800 flight narratives written by trained observers 4800 ratings of TEM countermeasure performance 4800 ratings of TEM countermeasure performance 17,500+ threats 17,500+ threats 12,500+ errors 12,500+ errors 2,400+ undesired aircraft states 2,400+ undesired aircraft states

5 Why is Archie here? To show what is possible when you build an archive of observational data based on TEM To show what is possible when you build an archive of observational data based on TEM To show what is possible when two TEM archives talk to each other To show what is possible when two TEM archives talk to each other

6 Archies Attributes 1. Archie is loyal and faithful 2. Archie is a persistent and diverse tracker 3. Archie plays well with others

7 After five years, Archie remains loyal to its aviation master The Archive is housed at UT The Archive is housed at UT All data are de-identified as to airline and individual All data are de-identified as to airline and individual No airline has been publicly identified No airline has been publicly identified No individual pilots can be or ever have been identified from the Archive No individual pilots can be or ever have been identified from the Archive

8 Archie is a good tracker ATC threats ATC threats Intentional Noncompliance Intentional Noncompliance Outstanding performance Outstanding performance

9 Tracking ATC threat chains Data are based on 10 LOSAs – 2426 Flights – 9450 threats 2350 ATC threats = ¼ of all threats = about 1 per flight 236 mismanaged ATC threats = 30% of all mismanaged threats 10% of all ATC threats were mismanaged

10 Mismanaged ATC threats 2426 Flights – 9450 threats 236 mismanaged ATC threats Challenging clearances (53%) Runway Changes (14%) Language difficulty (8%) Similar call signs (5%) ATC error (7%)

11 ATC threats -> Crew Errors 2426 Flights – 9450 threats 236 mismanaged ATC threats 280 errors Automation errors (20%) – wrong MCP/FCU altitude setting dialed Communication errors (27%) - misinterpret ATC instructions Aircraft handling (24%) – unintentional speed, lateral, vertical deviations Cross-verification errors (7%) Ground navigation errors (6%)

12 ATC threats -> Crew Errors -> UAS 2426 Flights – 9450 threats 236 mismanaged ATC threats 280 errors 56 Automation errors 76 Communication errors 67 Aircraft handling errors 19 Cross-verification errors 16 Ground navigation errors 2 UAS 58 UAS 5 UAS 9 UAS 21 UAS

13 ATC threats -> UAS 2426 Flights – 9450 threats 236 mismanaged ATC threats 280 errors 106 UAS Lateral deviation 17% Speed too high 14% Vertical deviation 10% Unstable Approach 10% Continued Landing 9% Incorrect Automation configuration 9% Taxiway/runway incursion 6% Speed too low 6% Incorrect Aircraft configuration 5% Other 14% Bottom line: 4% of flights had a UAS arising from an ATC threat that was mismanaged. ¾ occurred during descent/approach/land

14 Archie & Intentional Noncompliance Across 22 LOSAs, the average is 40% of flights with one or more noncompliance errors Across 22 LOSAs, the average is 40% of flights with one or more noncompliance errors Range: 23% - 90% Range: 23% - 90% Most common noncompliance errors: Most common noncompliance errors: checklist performed from memory / nonstandard checklist use checklist performed from memory / nonstandard checklist use failure to cross-verify MCP/FCU altitude alerter changes failure to cross-verify MCP/FCU altitude alerter changes PF makes own MCP/FCU changes PF makes own MCP/FCU changes Seems pretty trivial stuff, eh? Seems pretty trivial stuff, eh?

15 Noncompliance correlates with.. Based on 22 LOSAs, airlines with higher rates of intentional noncompliance also have more flights with: Mismanaged threats (r =.7) Mismanaged threats (r =.7) Mismanaged handling errors (r =.9) Mismanaged handling errors (r =.9) Undesired aircraft states (r =.9) Undesired aircraft states (r =.9) Mismanaged UASs (r =.8) Mismanaged UASs (r =.8) Noncompliance – a measure of safety culture? Noncompliance – a measure of safety culture? Archie was shocked…

16 Archie is a diverse tracker The richness and complexity of the data allow literally thousands of paths to be pursued The richness and complexity of the data allow literally thousands of paths to be pursued When did it happen, was it Captain or First Officer flying, how was it managed, what else was going on, what happened next, has it happened before, does it happen often, does it happen more on certain aircraft, does it happen at particular airports, etc. etc. etc. When did it happen, was it Captain or First Officer flying, how was it managed, what else was going on, what happened next, has it happened before, does it happen often, does it happen more on certain aircraft, does it happen at particular airports, etc. etc. etc. Archie can dig and dig… Archie can dig and dig…

17 Test Your Understanding A. Archie would find the flights that had above average operational complexity (4 or more threats), and that the crews managed very, very well. In the current Archive of 4800 flights, there are 102 of these defined flights. We can read the flight narratives to explore best practices… In the current Archive of 4800 flights, there are 102 of these defined flights. We can read the flight narratives to explore best practices… Q. What would you get if you asked Archie to find all the flights that have four or more threats, zero mismanaged threats, zero crew errors, zero UAS, and the observer rated the crew outstanding for overall effectiveness? GOOD BOY, ARCHIE!

18 Archie plays well with others Queries from / data- sharing with: ATC (NOSS group) ATC (NOSS group) IATA/ICAO (ITA) IATA/ICAO (ITA) Boeing Boeing Airlines Airlines Incident Reporting systems Incident Reporting systems Transport Safety Boards Transport Safety Boards

19 Archie plays well with others LOSA/TEM/NOSS data can complement other safety management data Flight Data Recorder is the aircraft perspective Flight Data Recorder is the aircraft perspective Incident reporting is the actors perspective Incident reporting is the actors perspective LOSA/NOSS is a neutral third party perspective LOSA/NOSS is a neutral third party perspective Weve never really had that before. Weve never really had that before. One Big Advantage: Pilot-ATC see the others world! One Big Advantage: Pilot-ATC see the others world!

20 Conclusions You have to be patient while the methodology matures and the Archive grows, but once in place: You can benchmark within and across facilities to determine a facilitys safety strengths and vulnerabilities You can benchmark within and across facilities to determine a facilitys safety strengths and vulnerabilities You can benchmark across the industry to determine systemic strengths, vulnerabilities, and best practices within the industry You can benchmark across the industry to determine systemic strengths, vulnerabilities, and best practices within the industry A TEM-based Archive would allow ATC to converse freely with the LOSA Archive on matters of mutual interest to pilots and controllers A TEM-based Archive would allow ATC to converse freely with the LOSA Archive on matters of mutual interest to pilots and controllers

21 The University of Texas Human Factors Research Project


Download ppt "A Glimpse at Archie: The LOSA Archive Ashleigh Merritt, Ph.D. The University of Texas Human Factors Research Project (UT) First ICAO Global Symposium on."

Similar presentations


Ads by Google