Presentation is loading. Please wait.

Presentation is loading. Please wait.

Accident Investigation Techniques and Methodologies

Similar presentations


Presentation on theme: "Accident Investigation Techniques and Methodologies"— Presentation transcript:

1 Accident Investigation Techniques and Methodologies
CRJ DFW September 4, No injuries when nose gear collapsed on turn toward terminal after a normal landing. Chuck DeJohn, D.O., M.P.H Federal Aviation Administration Civil Aerospace Medical Institute

2 Benner L. Starline Software Ltd. 1998
Benner L. Starline Software Ltd Safety Programs Hidden Defect: Accident Investigation

3 Frei R, Kingston J, Koornneef F, & Schallier P
Frei R, Kingston J, Koornneef F, & Schallier P. Investigation Tools in Context. JRC European Commission Institute for Energy Seminar. Investigation of Accidents. May Petten Netherlands. and on 4/23/04. Not every methodology or method is right for every investigation. A method or methodology should be selected based on the scale of the investigation and the depth of the investigation (level of abstraction). Frei R, Kingston J, Koornneef F, & Schallier P. Investigation Tools in Context. JRC European Commission Institute for Energy Seminar. Investigation of Accidents. May Petten Netherlands.

4 Methods vs. Methodology
Methodology: A system of principles, practices and body of procedures (methods) applied to a specific branch of knowledge. An overall approach to a field such as accident investigation. Examples: Adversarial, Commission, Events Reconstruction, Modeling, Simulation Method: A technique or tool. A regular, disciplined, systematic set of procedures used according to an underlying, detailed, logically ordered plan. Examples: Multi-linear Events Sequencing (MES), Fault Tree Analysis (FTA), Management Oversight and Risk Tree (MORT) Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC

5 Methods vs. Methodology Problems
Defined differently: By different authors By the same author in different articles By the same author in the same article! Methods and Methodologies are often used interchangeably Examples include Fault Tree Analysis (FTA) and Management Oversight and Risk Analysis Tree (MORT) Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3; Sometimes it is difficult to distinguish between using something as a method or a methodology, as we shall see. Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3;

6 Methodology Classification Schemes
Unstructured Adversarial Events Reconstruction Modeling Simulation Survey Archival Historical Experimental Case Study Common Sense Adversarial Engineering Statistical Symbolic Modeling vs. vs. Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris Ludwig Benner, Jr. on 3/23/04. on 4/24/04 Benner L. Methodological biases which undermine accident investigations Proceedings of the 18th Annual ISASI Seminar. Washington, D.C. on 4/24/04. The one on the left and the right are both from Benner of the NTSB. The one in the middle is from an ISASI paper. The Benner classification on the left is used more frequently in the literature and is the one we will use. Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers. Benner L. Methodological biases which undermine accident investigations Proceedings of the 18th Annual ISASI Seminar. Washington, D.C.

7 Unstructured Methodology
“Common Sense” or “Hunt-and-Peck” “Who, what, when, where, how and why?” Sequential ordering of events Explanation of the accident is acceptable if it “makes sense” Truth is determined by the investigator Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC. 1981 Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC. 1981

8 Adversarial Methodology
Rules of evidence and judicial procedures Opposing interests will bring out the truth Facts are gathered by the parties and informally tested by discussion against hypothesis for logic and consistency Reasoned conclusions logically drawn from technical evidence Examples U.S. Party System used by the NTSB Commission inquires (1) Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC (2) Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris Ludwig Benner, Jr. on March 23, 2004. Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris.

9 Events Reconstruction Methodology
Reconstruction of sequence of events (SOE): Physical evidence Witness interviews Speculation by investigator Methodology is not rigorous “Events” are undefined and highly variable Logic trees often culminate in event(s) selected by investigator without showing time relationships Probable cause (PC) often selected from one or more of the events Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris Ludwig Benner, Jr. on March 23, 2004. Can be very subjective. Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris.

10 Symbolic Modeling Methodologies
Pictorial representations of the SOE Fault Trees Failure selected and all possible factors that can contribute to the event are diagramed in the form of a tree Not always considered an overall methodology Examples: Logic Tree Analysis, Fault Tree Analysis (FTA), Management Oversight and Risk Tree (MORT), Multilinear Events Sequencing (MES) Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Harvey MD. Models for accident investigation Alberta Occupational Health and Safey Division. Occupational Health and Safey Division. Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris Ludwig Benner, Jr. on March 23, 2004. on 4/21/04 on 4/21/04 EM Jun 99. Appendix F. Use of Logic Trees in Probabilistic Seismic Hazard Analysis. on 4/22/04 Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Harvey MD. Models for accident investigation Alberta Occupational Health and Safey Division. Occupational Health and Safey Division. Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris. EM Jun 99. Appendix F. Use of Logic Trees in Probabilistic Seismic Hazard Analysis.

11 Simulation Methodologies
Reenactments that allow investigators to vary assumed events and asses effects of changes Formulate hypotheses Develop data where there are gaps Examples: Computerized modeling Scale modeling Use of actual aircraft/systems Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris Ludwig Benner, Jr. on March 23, 2004. Benner, L. Accident investigations – a case for new perceptions and methodologies. Archives of personal papers ex libris

12 Methodology Rankings Compare simultaneous investigations of the same accident using different methodologies Very resource intensive 1985 Benner Study: 17 U.S. Federal Government Agencies 10 evaluation criteria Benner L. Investigating investigation methodologies. Starline Software Ltd. Oakton, VA on 5/20/04. Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3; Benner L. Investigating investigation methodologies. Starline Software Ltd. Oakton, VA on 5/20/04. Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3;

13 Methodology Rankings Agencies Studied
Consumer Product Safety Commission Department of Agriculture Department of the Air Force Department of the Army Department of Energy Department of Labor Mine Safety and Health Administration Department of Labor Occupational Safety and Health Administration US Coast Guard Federal Highway Administration National Highway Traffic Safety Administration General Services Administration Library of Congress National Aeronautics and Space Administration National Institute of Occupational Safety and Health National Transportation Safety Board Navy Department Nuclear Regulatory Commission Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3; Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3;

14 Methodology Rankings Rating Criteria
Encouragement: Encourages harmonious participation. Independence: Produces unimpeachable results. Initiatives: Supports personal initiative. Discovery: Supports timely discovery of facts. Competence: Provides/improves employee competence. Standards: Provides for review of safety and health standards. Enforcement: Supports the enforcement program. States: Encourages states to take responsibility. Accuracy: Outputs can be tested for completeness, validity, logic and relevance. Closed Loop: Compatible with pre-investigation outputs. Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3; Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3;

15 Methodology Rankings Top Three
Event reconstruction Modeling MORT Fault Tree Adversarial Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3; Benner did not necessarily refer to the methodologies by these terms, but the descriptions fit. Benner, L. Rating accident models and investigation methodologies. J Safe Res Vol 12, No.3;

16 Accident Investigation Methods
Methods are Tools used by the investigator, not an overall system or branch of knowledge Most methods are sequencing tools – Reduce accidents to a collection of events using cause and effect relationships Fault Tree Analysis (FTA) Management Oversight and Risk Tree Analysis (MORT) Multilinear Events Sequencing (MES) Sequentially Timed Events Plotting (STEP) Events and Causal Factors Analysis (ECFA) Root Cause Analysis (RCA) Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC Benner, L. Methodology biases which undermine accident investigation. Proceedings of ISASI 18th Annual Seminar. Washington, DC

17 Accident Investigation Methods
To select the best method you should know: The name of the method you use now Which methods are available Which methods are better than others The outputs of the method you chose on 8/5/04

18 FTA Created at Bell Laboratories, refined by Boeing to analyze Minuteman missile problems and later adopted by DOD. Selected failure and all possible factors that can contribute are diagrammed into a tree. The accident is the “top event.” Top-down approach to determine how “top events” can be caused by individual or combined lower level failures. Events – Failures that lead to accidents. Gates – Ways failures combine to cause accidents. Useful for large accident investigations. Fault Tree Analysis Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Schiodtz K. Fault tree analysis in the application of accident analysis Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Schiodtz K. Fault tree analysis in the application of accident analysis

19 FTA Advantage Disadvantages
Conveniently represents main causes/factors of an accident Disadvantages No temporal relationships between events No ordering of events “Actors” not shown Fault Tree Analysis Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York.

20 FTA of Aircraft Runway Overrun Accident
Fault Tree Analysis Erickson, C.A. Accident Investigation Using EEFTA. Proceedings of the 18th International System Safety Conference. Seattle, Washington on 4/22/04. Aircraft runway overrun accident using Fault Tree Analysis. Erickson, C.A. Accident Investigation Using EEFTA. Proceedings of the 18th International System Safety Conference. Seattle, Washington

21 MORT Developed in the 1960s in response to the lack of accident investigation techniques that existed to support rigorous analysis Pre-designed, systematized logic tree in a generic graphical checklist format of approximately 1500 items Best suited to large complex accident Requires extensive training Management Oversight and Risk Tree Analysis Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. American Society of Safety Engineers. Northern Illinois University. TECH 438. MORT, Mini-MORT & PET on 4/24/04 Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. American Society of Safety Engineers. Northern Illinois University. TECH 438. MORT, Mini-MORT & PET

22 Mort Event Symbols Management Oversight and Risk Tree Analysis
on 4/24/04 American Society of Safety Engineers. Northern Illinois University. TECH 438. MORT, Mini-MORT & PET

23 Mort Logic Gates Management Oversight and Risk Tree Analysis
on 4/24/04 American Society of Safety Engineers. Northern Illinois University. TECH 438. MORT, Mini-MORT & PET

24 MORT Advantages Disadvantages
Systematically examines all possible causal factors Ideal when there is a shortage of expertise to ask the right questions Evaluates multiple causes Works well for complex accidents involving multiple systems Addresses root causes and contributory causes Looks beyond immediate causes including management/program factors Disadvantages Time consuming and tedious to use Requires extensive training Inappropriate for relatively simple accidents Can focus more on management than the accident event May lead to recommendations that are too broad (i.e. more training, more supervision) No temporal relationships between events Management Oversight and Risk Tree Analysis on 4/24/04 DOE. Accident Investigation Program. Section 7 – Analyzing Data. Oct on 4/24/04 Department of Energy Accident Investigation Program. Root cause analysis. January 19, 2001. on 5/24/04. Department of Energy. Accident Investigation Program. Section 7 – Analyzing Data. Oct 1999. Department of Energy Accident Investigation Program. Root cause analysis. January 19, 2001.

25 Abbreviated MORT Diagram
Management Oversight and Risk Tree Analysis PG Bishop, et al. Learning from incidents involving E/E/PE systems. Part 1 - Review of methods and industry practice HSE BOOKS. Norwich. on 4/21/04. LTA implies Less Than Adequate performance PG Bishop, et al. Learning from incidents involving E/E/PE systems. Part HSE Books. Norwich.

26 MES Time line chart of the accident process:Time line is displayed at the bottom of the chart and conditions and events are shown in logical order. Event = Actor + Action Event: Something of significance caused by an action. Actor: One who causes an event to occur. Does not have to be a person. Action: Acts performed by the actor. Multilinear Event Sequencing Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Harvey MD. Models for Accident Investigation. Workers Health, Safety and Compensation, Occupational Health and Safety Diovision. Alberta, Canada. April, Keong TH. Accident analysis techniques. Multilinear Events Sequencing. Benner L. Accident investigations: Multilinear events sequencing methods. J Safe Res. June Vol. 7. No. 2. Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Harvey MD. Models for Accident Investigation. Workers Health, Safety and Compensation, Occupational Health and Safety Diovision. Alberta, Canada. April, 1985. Keong TH. Accident analysis techniques. Multilinear Events Sequencing.

27 MES Accident sequence begins at to Accident sequence ends at tn
Stable situation is disturbed Beginning of the act which had to be detected, adapted, corrected, or otherwise changed for the course of events to have had a different outcome Accident sequence ends at tn Last consecutive harmful event connected directly with the accident Multilinear Event Sequencing Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York. Ferry TS. Modern accident investigation and analysis. Pp John Wiley & Sons. New York.

28 MES to = 11:01 tn = 11:02 Multilinear Event Sequencing
Benner L. Accident investigations: Multilinear events sequencing methods. J Safe Res. June Vol. 7. No. 2. Can = Conditions applying to actor A. Ena = Events caused by actor A. to = 11:01 tn = 11:02 Adapted from: Benner L. Accident investigations: Multilinear events sequencing methods. J Safe Res. June Vol. 7. No. 2.

29 MES Advantages Disadvantage Includes temporal relationship of events
Limits focus to the accident rather than focusing on management Has been called the best model available by some investigators Disadvantage Focuses almost exclusively on the accident and ignores management Multilinear Event Sequencing Harvey MD. Models for Accident Investigation. Workers Health, Safety and Compensation, Occupational Health and Safety Division. Alberta, Canada. April, Harvey MD. Models for Accident Investigation. Workers Health, Safety and Compensation, Occupational Health and Safety Division. Alberta, Canada. April, 1985.

30 STEP Developed by Hendrick and Benner in 1987
Refinement of the MES technique Each actor’s actions are traced from the start of an accident to the finish Actor + Action: Who (person or object) must do what to produce the next event Events are positioned along a timeline Causal links are represented by arrows connecting events Includes quality control with sufficient logic testing to assure consistency and validity Sequentially Timed Events Plotting Multiple Events Sequencing Livingston AS, Jackson G, & Priestly K. Root causes analysis: Literature review. Health and Safety Executive. Birchwood, Warrington on 5/24/04. NASA. QS/Safety and Risk Management Division. Procedures and guidelines for mishap reporting, investigating, and recordkeeping. NPG: June 2, on 5/27/04. Livingston AS, Jackson G, & Priestly K. Root causes analysis: Literature review. Health and Safety Executive. Birchwood, Warrington NASA. QS/Safety and Risk Management Division. Procedures and guidelines for mishap reporting, investigating, and recordkeeping. NPG: June 2, 2000.

31 STEP Sequentially Timed Events Plotting
Livingston AS, Jackson G, & Priestly K. Root causes analysis: Literature review. Health and Safety Executive. Birchwood, Warrington on 5/24/04. Livingston AS, Jackson G, & Priestly K. Root causes analysis: Literature review. Health and Safety Executive. Birchwood, Warrington.

32 ECFA Identifies causal factors for each significant event in an accident sequence Designed as a stand-alone technique but most effective when used with other methods (i.e. MORT, RCA) No “timeline” but temporal relationships are accounted for Events and Causal Factors Analysis Buys JR, Clark JL, Kingston-Howlett J, and Nelson HK. Events and causal factors analysis. Scientech, Inc. Idaho Falls, ID. August on 5/21/04 Buys JR, Clark JL, Kingston-Howlett J, and Nelson HK. Events and causal factors analysis. Scientech, Inc. Idaho Falls, ID. August 1995.

33 ECFA Evaluate events to determine significant events:
The accident would not have occurred if the significant event had not occurred The event deviated from what was planned or intended The event had unwanted consequences Determine the causal factors that allowed each significant event to occur: Who, why, what and how? Events and Causal Factors Analysis Definition of significant event is similar to the definition of root cause in RCA. Department of Energy Accident Investigation Program. Events and causal factors analysis. January 19, on 5/24/04. Department of Energy Accident Investigation Program. Events and causal factors analysis. January 19, 2001.

34 ECFA Example of Accident Chronology
Inspection of rudder PCU deleted from annual inspection Rudder PCU failure mode not identified Rudder PCU hydraulics contaminated Rudder hard-over in-flight 19:02:47 Crash 19:03:00 1994 September September 8 September 9 Event and Causal Factor Analysis Department of Energy Accident Investigation Program. 1/19/ on 5/21/04. Department of Energy Accident Investigation Program. 1/19/01.

35 ECFA Conditions for 1st Event
4 3 2 Rudder hard-over in-flight Crash Crew fails to respond to unusual attitude Crew fails to analyze unusual attitude Crew fails to recognize rudder problem Event and Causal Factor Analysis Department of Energy Accident Investigation Program. 1/19/ on 5/21/04. Department of Energy Accident Investigation Program. 1/19/01.

36 ECFA Conditions for 2nd Event
4 3 Rudder PCU hydraulics contaminated 1 Crash New maintenance personnel do not detect Hydraulic fluid becomes contaminated Change in maintenance services contract Event and Causal Factor Analysis Department of Energy Accident Investigation Program. 1/19/ on 5/21/04. Department of Energy Accident Investigation Program. 1/19/01.

37 ECFA Causal Factors for 1st Event
4 3 2 Rudder hard-over in-flight Crash Conditions Need for UA training unrecognized Potential need for recognizing rudder problems unrecognized Event and Causal Factor Analysis Department of Energy Accident Investigation Program. 1/19/ on 5/21/04. Department of Energy Accident Investigation Program. 1/19/01.

38 ECFA Causal Factors for 2nd Event
4 3 Rudder PCU contaminated 1 Crash Conditions Need to screen service contract provider unrecognized Potential for hydraulic fluid contamination unrecognized Event and Causal Factor Analysis Department of Energy Accident Investigation Program. 1/19/ on 5/21/04. Department of Energy Accident Investigation Program. 1/19/01.

39 ECFA Advantages Disadvantages
Temporal relationships of significant events preserved Ideal for multi-faceted problems with long or complex causal chain Causal factors for each significant event determined Recommendations easily arrived at from causal factors Helps to identify where deviations from acceptable procedures occurred Disadvantages Requires a broad perspective of the event to identify unrelated problems Time consuming Requires training/and or familiarity with the process Events and Causal Factors Analysis US Department of Energy, Office of Nuclear Energy, Office of Safety Policy and Standards. Root cause analysis guidance document. DOE-NE-STD Washington, D.C. February on 5/24/04. US Department of Energy, Office of Nuclear Energy, Office of Safety Policy and Standards. Root cause analysis guidance document. DOE-NE-STD Washington, D.C. February 1992.

40 RCA Root Causes are causal factors that, if corrected, would prevent the recurrence of the same or similar accident. Local Root Causes are specific deficiencies that, if corrected, would prevent the recurrence of the same accident. Systemic Root Causes are deficiencies in a management system that, if corrected, would prevent the occurrence of a class of similar accidents. Root Cause Analysis Department of Energy Accident Investigation Program. Root cause analysis. January 19, on 5/24/04. Department of Energy Accident Investigation Program. Root cause analysis. January 19, 2001.

41 RCA Root Cause Analysis (RCA) is a structured procedure to identify and evaluate the underlying causes of an accident to prevent a recurrence. Goal of RCA is not merely to determine the cause of an accident but to prevent it from occurring again. Root Cause Analysis NASA, Office of Safety and Mission Assurance, Chief Engineers Office. Root cause analysis overview. July on 5/24/04. Rimson IJ. Investigating “causes” and assigning “blame.” The Investigation Process Research Library. August on 5/24/04. Decision Systems, Inc. What is root cause analysis? Longview, TX on 5/24/04. NASA, Office of Safety and Mission Assurance, Chief Engineers Office. Root cause analysis overview. July 2003. Rimson IJ. Investigating “causes” and assigning “blame.” The Investigation Process Research Library. August 2003. Decision Systems, Inc. What is root cause analysis? Longview, TX

42 RCA Procedure Phase I: Clearly define the undesired outcome.
Phase II: Data Collection. Phase III: Assessment. Identify the problem and significance of the problem Identify the causes working back to the fundamental cause, which if corrected, would have prevented the accident (root cause) FTA MORT ECFA Root Cause Analysis NASA, Office of Safety and Mission Assurance, Chief Engineers Office. Root cause analysis overview. July on 5/24/04. Department of Energy Accident Investigation Program. Root cause analysis. January 19, on 5/24/04. NASA, Office of Safety and Mission Assurance, Chief Engineers Office. Root cause analysis overview. July 2003. Department of Energy Accident Investigation Program. Root cause analysis. January 19, 2001.

43 RCA Procedure Phase IV: Corrective actions for each identified cause to prevent recurrence. Phase V: Follow-up by determining if corrective action effectively prevents recurrence. Root Cause Analysis Department of Energy Accident Investigation Program. Root cause analysis. January 19, on 5/24/04. Department of Energy Accident Investigation Program. Root cause analysis. January 19, 2001.

44 RCA 5/20/04.

45 Conclusions Methodologies largely determined by organization
Methods may be selected Not all methods suitable for each accident Simplest method that yields the required results Frei R, Kingston J, Koornneef F, & Schallier P. Investigation Tools in Context. Noordwijk Risk Initiative Foundation. on 4/23/04. Not every methodology or method is right for every investigation. A method or methodology should be selected based on the scale of the investigation and the depth of the investigation (level of abstraction). Frei R, Kingston J, Koornneef F, & Schallier P. Investigation Tools in Context. Noordwijk Risk Initiative Foundation.

46


Download ppt "Accident Investigation Techniques and Methodologies"

Similar presentations


Ads by Google