Presentation on theme: "A Glimpse at Archie: The LOSA Archive"— Presentation transcript:
1 A Glimpse at Archie: The LOSA Archive Ashleigh Merritt, Ph.D.The University of Texas Human Factors Research Project (UT)First ICAO Global Symposiumon TEM & NOSS in ATC9-10 November, 2005, Luxembourg
2 What is Archie? “Archie” is the LOSA Archive A database of de-identified airline data which is maintained and updated by UTAirlines that do a LOSA with The LOSA Collaborative contribute their data to Archie for research purposesArchie provides the airline perspective
3 How old is Archie? Five LOSAs in five years to develop the methodology Data in the Archive starts in 2000Had to wait for Archie to grow before doing any benchmarking or industry-wide analyses.If we think of Archie as a dog, he would be a young bloodhound
4 How big is Archie? As of September 2005, Archie has data from 23 airline LOSAs4800 flights4800 flight narratives written by trained observers4800 ratings of TEM countermeasure performance17,500+ threats12,500+ errors2,400+ undesired aircraft states
5 Why is Archie here?To show what is possible when you build an archive of observational data based on TEMTo show what is possible when two TEM archives talk to each other
6 Archie’s Attributes Archie is loyal and faithful Archie is a persistent and diverse trackerArchie plays well with others
7 After five years, Archie remains loyal to its aviation master The Archive is housed at UTAll data are de-identified as to airline and individualNo airline has been publicly identifiedNo individual pilots can be or ever have been identified from the Archive
8 Archie is a good tracker ATC threatsIntentional NoncomplianceOutstanding performance
9 Tracking ATC threat chains Data are based on 10 LOSAs – 2426 Flights – 9450 threats2350 ATC threats= ¼ of all threats= about 1 per flight236 mismanaged ATC threats= 30% of all mismanaged threats10% of all ATC threats were mismanaged
13 236 mismanaged ATC threats ¾ occurred during descent/approach/land ATC threats -> UAS2426 Flights – 9450 threatsLateral deviation %Speed too high %Vertical deviation %Unstable Approach %Continued Landing %Incorrect Automation configuration %Taxiway/runway incursion %Speed too low %Incorrect Aircraft configuration % Other %280 errors236 mismanaged ATC threats106 UASBottom line:4% of flights had a UAS arising from an ATC threat that was mismanaged.¾ occurred during descent/approach/land
14 Archie & Intentional Noncompliance Across 22 LOSAs, the average is 40% of flights with one or more noncompliance errorsRange: 23% - 90%Most common noncompliance errors:checklist performed from memory / nonstandard checklist usefailure to cross-verify MCP/FCU altitude alerter changesPF makes own MCP/FCU changesSeems pretty trivial stuff, eh?
15 Noncompliance correlates with.. Based on 22 LOSAs, airlines with higher rates of intentional noncompliance also have more flights with:Mismanaged threats (r = .7)Mismanaged handling errors (r = .9)Undesired aircraft states (r = .9)Mismanaged UASs (r =.8)Noncompliance – a measure of safety culture?Archie was shocked…
16 Archie is a diverse tracker The richness and complexity of the data allow literally thousands of paths to be pursuedWhen did it happen, was it Captain or First Officer flying, how was it managed, what else was going on, what happened next, has it happened before, does it happen often, does it happen more on certain aircraft, does it happen at particular airports, etc. etc. etc.Archie can dig and dig…
17 Test Your Understanding Q. What would you get if you asked Archie to find all the flights that have four or more threats, zero mismanaged threats, zero crew errors, zero UAS, and the observer rated the crew “outstanding” for overall effectiveness?A. Archie would find the flights that had above average operational complexity (4 or more threats), and that the crews managed very, very well.In the current Archive of 4800 flights, there are 102 of these defined flights. We can read the flight narratives to explore best practices…GOOD BOY, ARCHIE!
18 Archie plays well with others Queries from / data-sharing with:ATC (NOSS group)IATA/ICAO (ITA)BoeingAirlinesIncident Reporting systemsTransport Safety Boards
19 Archie plays well with others LOSA/TEM/NOSS data can complement other safety management dataFlight Data Recorder is the aircraft perspectiveIncident reporting is the actor’s perspectiveLOSA/NOSS is a neutral third party perspectiveWe’ve never really had that before.One Big Advantage: Pilot-ATC see the others’ world!
20 ConclusionsYou have to be patient while the methodology matures and the Archive grows, but once in place:You can benchmark within and across facilities to determine a facility’s safety strengths and vulnerabilitiesYou can benchmark across the industry to determine systemic strengths, vulnerabilities, and best practices within the industryA TEM-based Archive would allow ATC to “converse” freely with the LOSA Archive on matters of mutual interest to pilots and controllers
21 The University of Texas Human Factors Research Project