Download presentation
Presentation is loading. Please wait.
1
HFACS ANALYSIS OF GENERAL AVIATION ACCIDENT DATA: IMPLICATIONS FOR AERONAUTICAL DECISION-MAKING
ASSESSING THE RELIABILITY OF THE HUMAN FACTORS ANALYSIS AND CLASSIFICAITON SYSTEM (HFACS) WITHIN THE CONTEXT OF GENERAL AVIATION D. A. WIEGMANN1 AND S. A. SHAPPELL2 1University of Illinois at Urbana-Champaign, Savoy, IL, 2Civil Aeromedical Institute, Oklahoma City, OK Purpose. The Human Factors Analysis and Classification System (HFACS) is a general human error framework for analyzing aviation accidents. Previous validation research has shown that HFACS produces reliable results when applied to both military and commercial aviation accident data. To date, however, the reliability of the system within the context of general aviation (GA) has yet to be examined. The purpose of the present study, therefore, was to assess the reliability of HFACS when applied to the analysis of GA accidents in an effort to further evaluate empirically the utility of the framework. Method. A comprehensive review of fatal GA airplane accidents (FAR Part 91) that occurred between 1995 and 1998 was conducted using databases maintained by the NTSB and FAA. All pilot-related causal factors associated with these accidents were independently coded by two certified pilots using the HFACS framework. Results. The reliability of the HFACS system, as indicated by the level of agreement between investigators, was assessed using Cohen’s Kappa. The Kappa indices obtained fell within the “Good” to “Excellent” range using conventional standards. Discussion. These findings corroborate previous research demonstrating the reliability of HFACS and underscore its utility as a general human error analysis tool across all categories of aviation operations. Douglas Wiegmann, Ph.D. University of Illinois Scott A. Shappell, Ph.D. Civil Aeromedical Institute FAA-99-G-006
2
Substandard Conditions of Operators PRECONDITIONS FOR UNSAFE ACTS
Errors Perceptual Skill-Based UNSAFE ACTS Decision Exceptional Routine Violations Inadequate Supervision Planned Inappropriate Operations Failed to Correct Problem Supervisory SUPERVISION Substandard Conditions of Operators PRECONDITIONS FOR UNSAFE ACTS Adverse Physiological States Physical/ Mental Limitations Adverse Mental States Personal Readiness Crew Resource Mismanagement Substandard Practices of Operators Resource Management Organizational Climate Process ORGANIZATIONAL INFLUENCES
3
Errors Perceptual Skill-Based UNSAFE ACTS Decision Exceptional Routine Violations Substandard Conditions of Operators PRECONDITIONS FOR UNSAFE ACTS Adverse Physiological States Physical/ Mental Limitations Adverse Mental States Personal Readiness Crew Resource Mismanagement Substandard Practices of Operators With few exceptions (e.g., flight instructors and flight training institutions), the top two tiers of HFACS (Unsafe Supervision and Organizational Influences) will not be populated when examining general aviation accidents. Consequently, only the bottom two tiers of HFACS were examined here. A more complete description of the data will be available at a later date.
4
FAR Part 91 General Aviation Accidents
Method A comprehensive review of fatal 14 CFR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA. A total of 2,391 accidents was identified for further analysis Only those accidents in which the accident investigation was completed and causal factors determined were included in this analysis. FAR Part 91 General Aviation Accidents The purpose of the final study was to explore the reliability of HFACS in classifying general aviation accidents. Albeit, the upper supervisory and organizational factors may be non-existent in many GA accidents, however, the lower tiers of the framework – preconditions and unsafe acts, are likely to be the same. A comprehensive review of fatal FAR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA. A total of 2,696 accidents were identified for further analysis. The 6,098 causal factors associated with these accidents were independently coded by five general aviation pilots, all certified flight instructors with flight hours ranging from 1,250 to 11,000.
5
FAR Part 91 General Aviation Accidents
Method The 5,893 causal factors associated with these 2,391 accidents were independently coded by five general aviation pilots This HFACS coding focused solely on the causal factors identified by the NTSB during the original accident investigation. No new cause factors were created during this error-coding process. FAR Part 91 General Aviation Accidents The purpose of the final study was to explore the reliability of HFACS in classifying general aviation accidents. Albeit, the upper supervisory and organizational factors may be non-existent in many GA accidents, however, the lower tiers of the framework – preconditions and unsafe acts, are likely to be the same. A comprehensive review of fatal FAR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA. A total of 2,696 accidents were identified for further analysis. The 6,098 causal factors associated with these accidents were independently coded by five general aviation pilots, all certified flight instructors with flight hours ranging from 1,250 to 11,000.
6
Method Coders: Five general aviation pilots/certified flight instructors Flight hours ranged from 1,250 to 11,000 Training: Four-hour workshop on HFACS Practice coding 20 accidents as a group Practice coding 50 accidents independently, followed by a review/consensus meeting Notes same as slide.
7
Sample of the Types of Human Error Typically Found
Aircraft Control Not Maintained Procedures/Directives Not Followed Abort Delayed Airspeed (VREF) Not Maintained APU Selected Proper Touchdown Point Misjudged Abort Above V1 Improper Airspeed (VMC) Not Maintained Autopilot Improper Use Of Complacency Control Interference Inadvertent Crew/Group Coordination Not Maintained Proper Touchdown Point Not Attained Airspeed Not Maintained Airspeed (VR) Improper Autopilot Inadvertent Deactivation Circuit Breaker Selected Compensation for Wind Conditions Not Possible Flare Improper Unsafe/Hazardous Condition Not Identified VFR Flight Into IMC Attempted Flight Into Adverse Weather Continued Hydraulic System Not Selected Inadequate Surveillance of Operation Proper Touchdown Point Not Possible Aborted Takeoff Delayed Airspeed (VLOF) Not Attained Airspeed Excessive Altimeter Setting Not Obtained Altitude Not Maintained Became Lost/Disoriented Checklist Not Complied With Crew/Group Coordination Not Performed Flaps Improper Use Of Flare Excessive Flight into Known Adverse Weather Initialed Go-Around Not Performed Identification of Aircraft Visually Delayed Inadequate Substantiation Process Visual Separation Not Maintained Minimum Descent Altitude Not Maintained Wheels Up Landing Inadvertent Aircraft Preflight Not Performed Aircraft Weight and Balance Misjudged Altimeter Not Used Checklist Inaccurate Compensation For Wind Conditions Inadequate Descent Excessive Distance Misjudged Flare Delayed Ground Loop/Swerve Intentional Remedial Action Delayed VFR Flight Into IMP Initiated Visual Lookout Not Maintained Abort Above V1 Performed Compensation for Wind Conditions Improper Directional Control Not Maintained Diverted Attention Ice/Frost Removal From Aircraft Inadequate IFR Procedure Improper Aircraft Control Not Possible Stall Inadvertent Inadequate Visual Lookout Lack of Familiarity With Aircraft Lack of Total Experience in Type of Aircraft Lowering of Flaps Performed Pressure VFR Flight Into IMC Inadvertent Aborted Takeoff Performed Communications Not Understood Emergency Procedure Not Followed Inadequate Weather Evaluation Nosewheel Steering Excessive Procedure Inadequate Rotation Excessive VFR Flight into IMC Continued Emergency Procedure Not Performed Lack of Familiarity with Geographic Area Level Off Not Attained Maintenance, Adjustment Improper Monitoring Inadequate Propeller Feathering Not Performed Remedial Action Not Possible Visual/Aural Perception Preflight Planning/Preparation Inadequate Aircraft Handling Improper Crew/Group Coordination Inadequate Spoiler Extension Not Performed Stall/Spin Inadvertent Airspeed (VREF) Not Attained Airspeed (VS) Not Maintained Go-Around Delayed Fatigue (Flight and Ground Schedule) Flight to Alternation Not Performed Operation with Known Deficiencies in Equipment Spoiler Extension Inadvertent Activation Supervision Inadequate Planning/Decision improper Raising of Flaps Improper In-Flight Planning/Decision Improper Overconfidence in Personal Ability Parking Brake Not Set Expectancy Flight Manuals Improper Use Of Wrong Taxi Route Selected Gear Extension Not Performed Weather Evaluation Inadequate Stall/Mush Encountered Parking Brakes Inadvertent Deactivation In-Flight Planning/Decision Poor Proper Glidepath Not Maintained Altitude Inadequate Conditions/Steps Insufficiently Defined Evacuation Improper Passenger Briefing Inadequate Spatial Disorientation Throttle/Power Control Improper Use Of Weather Evaluation Inaccurate Wrong Runway Selected Ice/Frost Removal From Aircraft Not Identified Planned Approach Poor Recovery from Bounced Landing Improper Planning/Decision Inadequate Aircraft Preflight Inadequate Checklist Inadequate Descent Inadvertent Generator Inadvertent Deactivation Touchdown Inadvertent Preflight Planning/Preparation Improper Compensation for Wind Conditions Misjudged Visual Illusion Uncontrolled Descent Proper Descent Rate Not Maintained Checklist Not Used Anti-Ice/Deice System Not Used Inadequate Monitoring Powerplant Controls Inadvertent Activation Traffic Advisory Not Identified Clearance Misjudged IFR Procedure Not Followed Inattentive Remedial Action Attempted Someone Goofed Improper Use of Preflight Briefing Service Descent Premature Proper Descent Rate Not Attained Airspeed Not Maintained (generic) Inadvertent Stall Visual Lookout Inadequate Ice/Frost Removal From Aircraft Nor Performed Information Insufficient Self-Induced Pressure Trim Setting Improper Flight Controls Improper Use Of Altitude/Clearance Not Maintained Maneuver Performed Preflight Planning/Preparation Poor Proper Altitude Not Maintained Flare Initiated Flight Advisories Not Followed Altitude/Clearance Inadequate Distance/Altitude Misjudged Inadequate Training Rotation Improper Unsuitable Terrain or Takeoff/Landing/Taxi Area VFR Procedures Inadequate Proper Alignment Not Possible Remedial Action Improper Flare Misjudged Proper Alignment Delayed Missed Approach Not Performed Proper Alignment Not Attained Lack of Total Experience in Type Operation Minimum Descent Altitude Below Miscellaneous Equipment Initiated Proper Alignment Not Maintained Supervision Improper Gear Down and Locked Not Verified Wind Information Misjudged Aircraft Weight and Balance Exceeded Aircraft Control-Uncontrolled Crew/Group Coordination Not Attained Checklist Not Followed Clearance Not Maintained
8
Procedure Each pilot was assigned 1/3 of the accidents for a given year. Independently coded NTSB cause factors (no new ones created) Randomly paired with a second pilot who coded the same set of accidents. Pilots met to compare codes and achieve consensus They were then assigned another 1/3 of the accidents for a particular year and randomly paired with another pilot. This process continued until all the accidents had been coded. Notes same as slide.
9
Results Reliability of Coding Process
On average, pilot agreed 79% of the time on how the causal factors should be coded using HFACS. These percentages varied only slight across the years of data analyzed in this study (range was 77% to 83% agreement). When overall agreement was corrected for chance using Cohen’s Kappa, the resulting index was .722, which is considered “good” by conventional standards. Notes same as slide.
10
Percentage of Accidents
Skill-based Errors Perceptual Decision Violations Percentage of Accidents Year * Incomplete Percentages do not add up to 100% The percentage of accidents associated with each form of unsafe act (skill-based errors, decision errors, perceptual errors, and violations) are presented for each year of the study ( ). Note that because each accident can have multiple causal factors associated with it, the percentages of accidents associated with each unsafe act will not equal 100%. Furthermore, because the percentage of one error form goes down does not necessarily mean that another must go up. In effect, they are independent of each other. An inspection of the figure illustrates several heretofore, unknown facts regarding fatal GA accidents. First, there has been little impact of efforts to date on specific types of human error associated with fatal GA accidents (i.e., the lines are essentially flat). This is in direct contrast to what has been reported previously in military and commercial aviation using HFACS. Second, skill-based errors have been associated with 4 out of every 5 accidents (80%) since These skill-based errors are primarily technique (stick-and-rudder) type errors indicating failures associated with training and currency/proficiency. Third, nearly 40% of all fatal GA accidents are associated with violations of the rules, and are typically the result of “continuing” flight into instrument meteorological conditions when authorized visual flight rules only. It is important to point out that these violations are “willful” departures from the rules and not simply inadvertent flight into the weather (classified as a decision error). Like violations, decision errors were also associated with nearly 40% of all fatal accidents, but perceptual errors (often due to visual illusions and spatial disorientation) were associated with less than 15% of all fatal accidents. It should be pointed out that many of our current intervention strategies and research efforts have been aimed at these last two error forms.
11
Results: Category of Cause-factors (%) Other (26.5%)
Conclusion Within the context of military, commercial, and general aviation, HFACS appears to be reliable. However, there is room for improvement. These improvements to the HFACS coding process could involve refinement of terms and methods for coding factors. Also of concern is the lack of detailed accident data contained in civilian databases, unlike that of military accident reports. Such conditions will limit the reliability of any system. The data reported here also highlight the importance of reassessing the reliability and other validation criteria of error frameworks when the type or category of data (military vs. commercial operations vs. GA) is changed. Finally, additional efforts are being made to validate the HFACS framework outside the flightdeck, including areas of aviation maintenance and air traffic control. Unsafe Acts (73.5%)
12
Results: Types of Unsafe Acts (%) % of Cause Factors
Conclusion Within the context of military, commercial, and general aviation, HFACS appears to be reliable. However, there is room for improvement. These improvements to the HFACS coding process could involve refinement of terms and methods for coding factors. Also of concern is the lack of detailed accident data contained in civilian databases, unlike that of military accident reports. Such conditions will limit the reliability of any system. The data reported here also highlight the importance of reassessing the reliability and other validation criteria of error frameworks when the type or category of data (military vs. commercial operations vs. GA) is changed. Finally, additional efforts are being made to validate the HFACS framework outside the flightdeck, including areas of aviation maintenance and air traffic control. N = 109 Type of Unsafe Act
13
Percentage of Cause Factors
Skill-based Errors Percentage of Cause Factors Violations Decision Errors Notes same as slide. Perceptual Errors Year
14
Variety of Decision Codes (N = 185)
inflight planning decision improper altitude inadequate judgment poor inflight planning decision poor inflight planning decision inadequate planning decision improper refueling not performed aborted take off not performed low altitude flight maneuver performed remedial action delayed aborted landing delayed wrong runway selected all available runway not used weather evaluation inadequate go around delayed preflight briefing service disregarded altitude improper aerobatics performed unsuitable terrain for take off landing selected preflight briefing service not used hazardous weather advisory disregarded procedure directives not followed vfr into imc inadvertent lift off premature aborted take off delayed preflight briefing service not obtained weather forecast disregarded go around not performed altitude low inflight planning decision delayed anti ice de ice system not used proper altitude not selected flight to alternate destination not performed weather evaluation misjudged weather evaluation improper weather evaluation poor missed approach not performed emergency procedure simulated vfr into imc attempted pull up excessive pull up delayed planned approach improper planned approach poor taxi speed excessive ostentatious display fuel supply inadequate carburetor heat improper use of flight into adverse weather continued flight into adverse weather inadvertent low altitude flight maneuver intentional flight into adverse weather initiated emergency procedure delayed flight advisory disregarded weather forecast not obtained maneuver excessive remedial action inadequate stall spin initiated missed approach delayed gear retraction not performed low pass performed planning decision poor carburetor heat not used flight to alternate destination delayed updating of recorded weather info not obtained pull up performed go around attempted emergency procedure not performed altitude clearance inadequate improper decision stall spin performed maneuver performed remedial action not performed refueling improper ac handling improper proper altitude not maintained
15
Top 10 Decision Errors Decision Error Frequency
Poor In-flight Planning 184 (25.8%) Wrong Altitude Selection 78 (10.9%) Over Commitment to Plan 50 (7.0%) Poor Judgment Poor Flight Planning 36 (5.0%) Inaccurate Weather Evaluation 22 (3.1%) Refueling Not Performed 21 (2.9%) Inadequate Preflight Briefing 20 (2.8%) Improper Remedial Action 16 (2.2%) Wrong Maneuver 15 (2.1%)
16
Variety of Violation Codes (N = 115)
ac control exceeded ac control not maintained ac unattended engine running intentional ac weight balance continued ac weight balance disregarded ac weight balance exceeded ac weight balance excessive ac weight balance improper act clearance not complied aerobatics attempted aerobatics improper aerobatics initiated aerobatics intentional aerobatics performed air speed exceeded altitude clearance inadequate altitude disregarded altitude inadequate altitude low ATC clearance not followed attitude indicator not available buzzing intentional buzzing performed certification improper for flight decision height disregarded decision height not complied with decision height not maintained decision height not used descent height disregarded design stress limits of ac exceeded dispatch procedures not followed other govt personnel external navigation lights not used flight into adverse weather flight into adverse weather attempted flight into adverse weather continued flight into adverse weather improper flight into adverse weather inadvertent flight into adverse weather initiated flight into adverse weather intentional flight into adverse weather performed flight into adverse weather selected flight manuals disregarded flight navigation instruments inadequate fuel supply inadequate hazardous weather advisory disregarded ice frost removal from ac improper ifr procedure improper ifr procedure not followed impairment alcohol impairment drugs inflight briefing service not used inflight planning decision improper information insufficient designated examiner information insufficient pic loading of cargo improper low altitude flight maneuver attempted low altitude flight maneuver intentional low altitude flight maneuver performed low pass intentional low pass performed maintenance annual inspection not complied with maintenance annual inspection not performed maintenance design changes improper pic maintenance installation improper maintenance major alteration improper pic maintenance major repair improper pic maintenance service bulletin not complied with maneuver attempted minimum descent altitude below minimum descent altitude disregarded minimum descent altitude not complied with minimum descent altitude not maintained minimum descent altitude not obtained/maintained
17
Top 10 Violations Violation Frequency VFR into IMC 262 (28.7%)
Flight into Adverse Weather 157 (17.2%) Stress Limits Exceeded 87 (9.5%) IFR Procedure Not Followed 53 (5.8%) Weight and Balance Exceeded 49 (5.4%) Aerobatics Performed 48 (5.3%) Ostentatious Display/Buzzing Procedure/Directives Not Followed 46 (5.0%) Operating With Known Deficiencies 40 (4.4%) Min. Descent Altitude not Complied with 26 (2.9%)
18
Top 5 Decision Errors/Violations
Causal Factor Frequency VFR into IMC 262 (16.1%) Poor In-flight Planning 184 (11.3%) Flight into Adverse Weather 157 (9.7%) Stress Limits Exceeded 87 (5.4%) Wrong Altitude Selection 78 (4.8%)
19
Conclusions Top factors involve weather-related issues
These factors have been explore by the Wx JSAT FAA is sponsoring research to empirically explore these factors (go beyond “expert opinion”) When decision errors are addressed separately from violations: Flight planning and management become the top issues This may be more inline with the traditional conceptualization of ADM Notes same as slide.
Similar presentations
