Presentation on theme: "HFACS ANALYSIS OF GENERAL AVIATION ACCIDENT DATA: IMPLICATIONS FOR AERONAUTICAL DECISION-MAKING ASSESSING THE RELIABILITY OF THE HUMAN FACTORS ANALYSIS."— Presentation transcript:
1 HFACS ANALYSIS OF GENERAL AVIATION ACCIDENT DATA: IMPLICATIONS FOR AERONAUTICAL DECISION-MAKING ASSESSING THE RELIABILITY OF THE HUMAN FACTORS ANALYSIS AND CLASSIFICAITON SYSTEM (HFACS) WITHIN THE CONTEXT OF GENERAL AVIATIOND. A. WIEGMANN1 AND S. A. SHAPPELL21University of Illinois at Urbana-Champaign, Savoy, IL, 2Civil Aeromedical Institute, Oklahoma City, OKPurpose. The Human Factors Analysis and Classification System (HFACS) is a general human error framework for analyzing aviation accidents. Previous validation research has shown that HFACS produces reliable results when applied to both military and commercial aviation accident data. To date, however, the reliability of the system within the context of general aviation (GA) has yet to be examined. The purpose of the present study, therefore, was to assess the reliability of HFACS when applied to the analysis of GA accidents in an effort to further evaluate empirically the utility of the framework. Method. A comprehensive review of fatal GA airplane accidents (FAR Part 91) that occurred between 1995 and 1998 was conducted using databases maintained by the NTSB and FAA. All pilot-related causal factors associated with these accidents were independently coded by two certified pilots using the HFACS framework. Results. The reliability of the HFACS system, as indicated by the level of agreement between investigators, was assessed using Cohen’s Kappa. The Kappa indices obtained fell within the “Good” to “Excellent” range using conventional standards. Discussion. These findings corroborate previous research demonstrating the reliability of HFACS and underscore its utility as a general human error analysis tool across all categories of aviation operations.Douglas Wiegmann, Ph.D.University of IllinoisScott A. Shappell, Ph.D.Civil Aeromedical InstituteFAA-99-G-006
2 Substandard Conditions of Operators PRECONDITIONS FOR UNSAFE ACTS ErrorsPerceptualSkill-BasedUNSAFEACTSDecisionExceptionalRoutineViolationsInadequateSupervisionPlannedInappropriateOperationsFailed toCorrectProblemSupervisorySUPERVISIONSubstandard Conditions of OperatorsPRECONDITIONSFORUNSAFE ACTSAdverse Physiological StatesPhysical/MentalLimitationsAdverse Mental StatesPersonal ReadinessCrew Resource MismanagementSubstandard Practices of OperatorsResourceManagementOrganizationalClimateProcessORGANIZATIONALINFLUENCES
3 ErrorsPerceptualSkill-BasedUNSAFEACTSDecisionExceptionalRoutineViolationsSubstandard Conditions of OperatorsPRECONDITIONSFORUNSAFE ACTSAdverse Physiological StatesPhysical/MentalLimitationsAdverse Mental StatesPersonal ReadinessCrew Resource MismanagementSubstandard Practices of OperatorsWith few exceptions (e.g., flight instructors and flight training institutions), the top two tiers of HFACS (Unsafe Supervision and Organizational Influences) will not be populated when examining general aviation accidents. Consequently, only the bottom two tiers of HFACS were examined here. A more complete description of the data will be available at a later date.
4 FAR Part 91 General Aviation Accidents MethodA comprehensive review of fatal 14 CFR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA.A total of 2,391 accidents was identified for further analysisOnly those accidents in which the accident investigation was completed and causal factors determined were included in this analysis.FAR Part 91 General Aviation AccidentsThe purpose of the final study was to explore the reliability of HFACS in classifying general aviation accidents. Albeit, the upper supervisory and organizational factors may be non-existent in many GA accidents, however, the lower tiers of the framework – preconditions and unsafe acts, are likely to be the same. A comprehensive review of fatal FAR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA. A total of 2,696 accidents were identified for further analysis. The 6,098 causal factors associated with these accidents were independently coded by five general aviation pilots, all certified flight instructors with flight hours ranging from 1,250 to 11,000.
5 FAR Part 91 General Aviation Accidents MethodThe 5,893 causal factors associated with these 2,391 accidents were independently coded by five general aviation pilotsThis HFACS coding focused solely on the causal factors identified by the NTSB during the original accident investigation.No new cause factors were created during this error-coding process.FAR Part 91 General Aviation AccidentsThe purpose of the final study was to explore the reliability of HFACS in classifying general aviation accidents. Albeit, the upper supervisory and organizational factors may be non-existent in many GA accidents, however, the lower tiers of the framework – preconditions and unsafe acts, are likely to be the same. A comprehensive review of fatal FAR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA. A total of 2,696 accidents were identified for further analysis. The 6,098 causal factors associated with these accidents were independently coded by five general aviation pilots, all certified flight instructors with flight hours ranging from 1,250 to 11,000.
6 MethodCoders:Five general aviation pilots/certified flight instructorsFlight hours ranged from 1,250 to 11,000Training:Four-hour workshop on HFACSPractice coding 20 accidents as a groupPractice coding 50 accidents independently, followed by a review/consensus meetingNotes same as slide.
7 Sample of the Types of Human Error Typically Found Aircraft Control Not MaintainedProcedures/Directives Not FollowedAbort DelayedAirspeed (VREF) Not MaintainedAPU SelectedProper Touchdown Point MisjudgedAbort Above V1 ImproperAirspeed (VMC) Not MaintainedAutopilot Improper Use OfComplacencyControl Interference InadvertentCrew/Group Coordination Not MaintainedProper Touchdown Point Not AttainedAirspeed Not MaintainedAirspeed (VR) ImproperAutopilot Inadvertent DeactivationCircuit Breaker SelectedCompensation for Wind Conditions Not PossibleFlare ImproperUnsafe/Hazardous Condition Not IdentifiedVFR Flight Into IMC AttemptedFlight Into Adverse Weather ContinuedHydraulic System Not SelectedInadequate Surveillance of OperationProper Touchdown Point Not PossibleAborted Takeoff DelayedAirspeed (VLOF) Not AttainedAirspeed ExcessiveAltimeter Setting Not ObtainedAltitude Not MaintainedBecame Lost/DisorientedChecklist Not Complied WithCrew/Group Coordination Not PerformedFlaps Improper Use OfFlare ExcessiveFlight into Known Adverse Weather InitialedGo-Around Not PerformedIdentification of Aircraft Visually DelayedInadequate Substantiation ProcessVisual Separation Not MaintainedMinimum Descent Altitude Not MaintainedWheels Up Landing InadvertentAircraft Preflight Not PerformedAircraft Weight and Balance MisjudgedAltimeter Not UsedChecklist InaccurateCompensation For Wind Conditions InadequateDescent ExcessiveDistance MisjudgedFlare DelayedGround Loop/Swerve IntentionalRemedial Action DelayedVFR Flight Into IMP InitiatedVisual Lookout Not MaintainedAbort Above V1 PerformedCompensation for Wind Conditions ImproperDirectional Control Not MaintainedDiverted AttentionIce/Frost Removal From Aircraft InadequateIFR Procedure ImproperAircraft Control Not PossibleStall InadvertentInadequate Visual LookoutLack of Familiarity With AircraftLack of Total Experience in Type of AircraftLowering of Flaps PerformedPressureVFR Flight Into IMC InadvertentAborted Takeoff PerformedCommunications Not UnderstoodEmergency Procedure Not FollowedInadequate Weather EvaluationNosewheel Steering ExcessiveProcedure InadequateRotation ExcessiveVFR Flight into IMC ContinuedEmergency Procedure Not PerformedLack of Familiarity with Geographic AreaLevel Off Not AttainedMaintenance, Adjustment ImproperMonitoring InadequatePropeller Feathering Not PerformedRemedial Action Not PossibleVisual/Aural PerceptionPreflight Planning/Preparation InadequateAircraft Handling ImproperCrew/Group Coordination InadequateSpoiler Extension Not PerformedStall/Spin InadvertentAirspeed (VREF) Not AttainedAirspeed (VS) Not MaintainedGo-Around DelayedFatigue (Flight and Ground Schedule)Flight to Alternation Not PerformedOperation with Known Deficiencies in EquipmentSpoiler Extension Inadvertent ActivationSupervision InadequatePlanning/Decision improperRaising of Flaps ImproperIn-Flight Planning/Decision ImproperOverconfidence in Personal AbilityParking Brake Not SetExpectancyFlight Manuals Improper Use OfWrong Taxi Route SelectedGear Extension Not PerformedWeather Evaluation InadequateStall/Mush EncounteredParking Brakes Inadvertent DeactivationIn-Flight Planning/Decision PoorProper Glidepath Not MaintainedAltitude InadequateConditions/Steps Insufficiently DefinedEvacuation ImproperPassenger Briefing InadequateSpatial DisorientationThrottle/Power Control Improper Use OfWeather Evaluation InaccurateWrong Runway SelectedIce/Frost Removal From Aircraft Not IdentifiedPlanned Approach PoorRecovery from Bounced Landing ImproperPlanning/Decision InadequateAircraft Preflight InadequateChecklist InadequateDescent InadvertentGenerator Inadvertent DeactivationTouchdown InadvertentPreflight Planning/Preparation ImproperCompensation for Wind Conditions MisjudgedVisual IllusionUncontrolled DescentProper Descent Rate Not MaintainedChecklist Not UsedAnti-Ice/Deice System Not UsedInadequate MonitoringPowerplant Controls Inadvertent ActivationTraffic Advisory Not IdentifiedClearance MisjudgedIFR Procedure Not FollowedInattentiveRemedial Action AttemptedSomeone GoofedImproper Use of Preflight Briefing ServiceDescent PrematureProper Descent Rate Not AttainedAirspeed Not Maintained (generic)Inadvertent StallVisual Lookout InadequateIce/Frost Removal From Aircraft Nor PerformedInformation InsufficientSelf-Induced PressureTrim Setting ImproperFlight Controls Improper Use OfAltitude/Clearance Not MaintainedManeuver PerformedPreflight Planning/Preparation PoorProper Altitude Not MaintainedFlare InitiatedFlight Advisories Not FollowedAltitude/Clearance InadequateDistance/Altitude MisjudgedInadequate TrainingRotation ImproperUnsuitable Terrain or Takeoff/Landing/Taxi AreaVFR Procedures InadequateProper Alignment Not PossibleRemedial Action ImproperFlare MisjudgedProper Alignment DelayedMissed Approach Not PerformedProper Alignment Not AttainedLack of Total Experience in Type OperationMinimum Descent Altitude BelowMiscellaneous Equipment InitiatedProper Alignment Not MaintainedSupervision ImproperGear Down and Locked Not VerifiedWind Information MisjudgedAircraft Weight and Balance ExceededAircraft Control-UncontrolledCrew/Group Coordination Not AttainedChecklist Not FollowedClearance Not Maintained
8 ProcedureEach pilot was assigned 1/3 of the accidents for a given year.Independently coded NTSB cause factors (no new ones created)Randomly paired with a second pilot who coded the same set of accidents.Pilots met to compare codes and achieve consensusThey were then assigned another 1/3 of the accidents for a particular year and randomly paired with another pilot.This process continued until all the accidents had been coded.Notes same as slide.
9 Results Reliability of Coding Process On average, pilot agreed 79% of the time on how the causal factors should be coded using HFACS.These percentages varied only slight across the years of data analyzed in this study (range was 77% to 83% agreement).When overall agreement was corrected for chance using Cohen’s Kappa, the resulting index was .722, which is considered “good” by conventional standards.Notes same as slide.
10 Percentage of Accidents Skill-basedErrorsPerceptualDecisionViolationsPercentage of AccidentsYear* IncompletePercentages do not add up to 100%The percentage of accidents associated with each form of unsafe act (skill-based errors, decision errors, perceptual errors, and violations) are presented for each year of the study ( ). Note that because each accident can have multiple causal factors associated with it, the percentages of accidents associated with each unsafe act will not equal 100%. Furthermore, because the percentage of one error form goes down does not necessarily mean that another must go up. In effect, they are independent of each other.An inspection of the figure illustrates several heretofore, unknown facts regarding fatal GA accidents. First, there has been little impact of efforts to date on specific types of human error associated with fatal GA accidents (i.e., the lines are essentially flat). This is in direct contrast to what has been reported previously in military and commercial aviation using HFACS. Second, skill-based errors have been associated with 4 out of every 5 accidents (80%) since These skill-based errors are primarily technique (stick-and-rudder) type errors indicating failures associated with training and currency/proficiency. Third, nearly 40% of all fatal GA accidents are associated with violations of the rules, and are typically the result of “continuing” flight into instrument meteorological conditions when authorized visual flight rules only. It is important to point out that these violations are “willful” departures from the rules and not simply inadvertent flight into the weather (classified as a decision error). Like violations, decision errors were also associated with nearly 40% of all fatal accidents, but perceptual errors (often due to visual illusions and spatial disorientation) were associated with less than 15% of all fatal accidents. It should be pointed out that many of our current intervention strategies and research efforts have been aimed at these last two error forms.
11 Results: Category of Cause-factors (%) Other (26.5%) ConclusionWithin the context of military, commercial, and general aviation, HFACS appears to be reliable. However, there is room for improvement. These improvements to the HFACS coding process could involve refinement of terms and methods for coding factors. Also of concern is the lack of detailed accident data contained in civilian databases, unlike that of military accident reports. Such conditions will limit the reliability of any system.The data reported here also highlight the importance of reassessing the reliability and other validation criteria of error frameworks when the type or category of data (military vs. commercial operations vs. GA) is changed.Finally, additional efforts are being made to validate the HFACS framework outside the flightdeck, including areas of aviation maintenance and air traffic control.Unsafe Acts (73.5%)
12 Results: Types of Unsafe Acts (%) % of Cause Factors ConclusionWithin the context of military, commercial, and general aviation, HFACS appears to be reliable. However, there is room for improvement. These improvements to the HFACS coding process could involve refinement of terms and methods for coding factors. Also of concern is the lack of detailed accident data contained in civilian databases, unlike that of military accident reports. Such conditions will limit the reliability of any system.The data reported here also highlight the importance of reassessing the reliability and other validation criteria of error frameworks when the type or category of data (military vs. commercial operations vs. GA) is changed.Finally, additional efforts are being made to validate the HFACS framework outside the flightdeck, including areas of aviation maintenance and air traffic control.N = 109Type of Unsafe Act
13 Percentage of Cause Factors Skill-based ErrorsPercentage of Cause FactorsViolationsDecision ErrorsNotes same as slide.Perceptual ErrorsYear
14 Variety of Decision Codes (N = 185) inflight planning decision improperaltitude inadequatejudgment poorinflight planning decision poorinflight planning decision inadequateplanning decision improperrefueling not performedaborted take off not performedlow altitude flight maneuver performedremedial action delayedaborted landing delayedwrong runway selectedall available runway not usedweather evaluation inadequatego around delayedpreflight briefing service disregardedaltitude improperaerobatics performedunsuitable terrain for take off landing selectedpreflight briefing service not usedhazardous weather advisory disregardedprocedure directives not followedvfr into imc inadvertentlift off prematureaborted take off delayedpreflight briefing service not obtainedweather forecast disregardedgo around not performedaltitude lowinflight planning decision delayedanti ice de ice system not usedproper altitude not selectedflight to alternate destination not performedweather evaluation misjudgedweather evaluation improperweather evaluation poormissed approach not performedemergency procedure simulatedvfr into imc attemptedpull up excessivepull up delayedplanned approach improperplanned approach poortaxi speed excessiveostentatious displayfuel supply inadequatecarburetor heat improper use offlight into adverse weather continuedflight into adverse weather inadvertentlow altitude flight maneuver intentionalflight into adverse weather initiatedemergency procedure delayedflight advisory disregardedweather forecast not obtainedmaneuver excessiveremedial action inadequatestall spin initiatedmissed approach delayedgear retraction not performedlow pass performedplanning decision poorcarburetor heat not usedflight to alternate destination delayedupdating of recorded weather info not obtainedpull up performedgo around attemptedemergency procedure not performedaltitude clearance inadequateimproper decisionstall spin performedmaneuver performedremedial action not performedrefueling improperac handling improperproper altitude not maintained
15 Top 10 Decision Errors Decision Error Frequency Poor In-flight Planning184 (25.8%)Wrong Altitude Selection78 (10.9%)Over Commitment to Plan50 (7.0%)Poor JudgmentPoor Flight Planning36 (5.0%)Inaccurate Weather Evaluation22 (3.1%)Refueling Not Performed21 (2.9%)Inadequate Preflight Briefing20 (2.8%)Improper Remedial Action16 (2.2%)Wrong Maneuver15 (2.1%)
16 Variety of Violation Codes (N = 115) ac control exceededac control not maintainedac unattended engine running intentionalac weight balance continuedac weight balance disregardedac weight balance exceededac weight balance excessiveac weight balance improperact clearance not compliedaerobatics attemptedaerobatics improperaerobatics initiatedaerobatics intentionalaerobatics performedair speed exceededaltitude clearance inadequatealtitude disregardedaltitude inadequatealtitude lowATC clearance not followedattitude indicator not availablebuzzing intentionalbuzzing performedcertification improper for flightdecision height disregardeddecision height not complied withdecision height not maintaineddecision height not useddescent height disregardeddesign stress limits of ac exceededdispatch procedures not followed other govt personnelexternal navigation lights not usedflight into adverse weatherflight into adverse weather attemptedflight into adverse weather continuedflight into adverse weather improperflight into adverse weather inadvertentflight into adverse weather initiatedflight into adverse weather intentionalflight into adverse weather performedflight into adverse weather selectedflight manuals disregardedflight navigation instruments inadequatefuel supply inadequatehazardous weather advisory disregardedice frost removal from ac improperifr procedure improperifr procedure not followedimpairment alcoholimpairment drugsinflight briefing service not usedinflight planning decision improperinformation insufficient designated examinerinformation insufficient picloading of cargo improperlow altitude flight maneuver attemptedlow altitude flight maneuver intentionallow altitude flight maneuver performedlow pass intentionallow pass performedmaintenance annual inspection not complied withmaintenance annual inspection not performedmaintenance design changes improper picmaintenance installation impropermaintenance major alteration improper picmaintenance major repair improper picmaintenance service bulletin not complied withmaneuver attemptedminimum descent altitude belowminimum descent altitude disregardedminimum descent altitude not complied withminimum descent altitude not maintainedminimum descent altitude not obtained/maintained
17 Top 10 Violations Violation Frequency VFR into IMC 262 (28.7%) Flight into Adverse Weather157 (17.2%)Stress Limits Exceeded87 (9.5%)IFR Procedure Not Followed53 (5.8%)Weight and Balance Exceeded49 (5.4%)Aerobatics Performed48 (5.3%)Ostentatious Display/BuzzingProcedure/Directives Not Followed46 (5.0%)Operating With Known Deficiencies40 (4.4%)Min. Descent Altitude not Complied with26 (2.9%)
18 Top 5 Decision Errors/Violations Causal FactorFrequencyVFR into IMC262 (16.1%)Poor In-flight Planning184 (11.3%)Flight into Adverse Weather157 (9.7%)Stress Limits Exceeded87 (5.4%)Wrong Altitude Selection78 (4.8%)
19 Conclusions Top factors involve weather-related issues These factors have been explore by the Wx JSATFAA is sponsoring research to empirically explore these factors (go beyond “expert opinion”)When decision errors are addressed separately from violations:Flight planning and management become the top issuesThis may be more inline with the traditional conceptualization of ADMNotes same as slide.