1. 1

2. 2 How to Read and “Right” a Causal Factors Analysis Report ISM Workshop – Feedback and Improvement Session B-2 Richard S. Hartley, Ph.D., P.E. November 28, 2007

3. 3 What We Are Going To Cover HRO Attributes Organizational Causes of Events Demystifying Causal Factors Analysis Black Magic of CFA How to Read and “Right” a CFA Report CFA Feedback on the HRO

4. 4 HRO Attributes Accidents can be avoided 1.Leadership Safety Objectives  Safety held high as priority  Provide clear operational goals  Willing to pay for redundancy and continuous training 2.Redundancy  Independent barriers reduce the fatal error by one person 3.Decentralization, Culture, Continuity  Worker calls the shots  Worker calls the shots like management wants him to  Worker stays proficient through continous training and work  Worker maintains his awareness by expecting the unexpected 4.Organizational Learning  Adjust through trial and error  Learn from past mistakes  Simulate where practice not feasible Scott Sagan, The Limits of Safety

5. 5 How Organizational Accidents Occur Vision, Beliefs, & Values Latent Organizational Weaknesses Mission Goals Policies Processes Programs Different Vision, Beliefs, & Values Event Flawed Defenses Human Errors Initiating Action

6. 6 I/O to Achieve Desired Result of Causal Factor Investigations 3 Process Outputs To Achieve Results: Safety Culture Insight Corrective Actions Lessons-To-Be Learned Unique Feature Desired Results: Prevent Recurrence of Events Learn as an Organization Desired Results: Prevent Recurrence of Events Learn as an Organization Criteria to grade CFA Reports “READ” Key Attributes of HRO Criteria to drive CFA Investigations “RIGHT” Process Input Low consequence, information rich events

7. 7 Process Input - Information-Rich Events Significant gap between “work-as- imagined” (planned) vs. “work-as-done” (actual efforts)* New type of event/error not afforded protection by existing systems Significantly worse consequences could have occurred * The Field Guide to Understanding Human Error – Sidney Dekker

8. 8 “Information Rich” Determination Matrix Co-developed Bill Corcoran & Richard Hartley 8-21-07 CriteriaEvidenceRating Gap between “work as imagined” vs.” work-as-done” 10 indicates significant gap 0 indicates insignificant gap 0-10 Precursor (what is this event a precursor to)  10 indicates very significant to a follow-on event  0 indicates not significant in leading to a follow-on event 0-10 Extent  10 indicates could happen at other places on Plant-site  0 indicates that problem is contained within area of investigation (unique event) 0-10 Barrier Issues  10 indicates many important barriers had problems (missing or failed)  0 indicates that most barriers remained effective 0-10 Fragility of remaining barriers  10 indicates that remaining barriers were few and fragile or not known to worker  0 indicates remaining barriers effective & robust 0-10 Campaign Issues  10 indicates important issue to customer, regulators, corporation, etc.  0 not of high importance to customer, regulators, corporation, etc. 0-10 Overall Figure of Significance  To determine overall figure of significance, take the total and divide by 6 (criteria)  Ranking: 10 – 7  significant event (CFA Investigation) 6 – 4  important event (CA/MP or streamline cause analysis) 3 – 1  less significant event (Apparent Cause) 0-10

9. 9 Process Output – Safety Culture Insight Types of Safety Culture: Default Culture (what you get if you do nothing) Managed Culture (push organization to desired state) Safety Culture Can Be Characterized* Mental Content (shared values, beliefs) Norms (commonly encountered behavior) Institutions (programs - QA, Safety, etc.) Physical Items (procedures, PPE, etc.) Safety Culture Can Be Evaluated** Problem Identification & Resolution Human Performance Safety Conscious Work Environment Organizational Issue Harmful Attributes of Safety Culture Can Be Modified * © 2007 W. R. Corcoran, NSRC Corporation ** NRC – Regulatory Issue Summary 2006-13 Co-developed Bill Corcoran & Richard Hartley 8-21-07

10. 10 Process Output - Corrective Actions Required for Causal Factors Human Performance Error Precursors Flawed Barriers Extraneous Conditions Adverse to Quality Modify Based on Extent of Problem Confidence in Corrective Actions Requires Confidence in Factors Confidence in Factors Requires They Explain Nature, Magnitude or Timing of Consequences Consequences Selection of “right” consequences key to “right” factors Types: Actual, Expected, Potential Confidence in Consequences, Extent, ECAQ Requires: Complete collection and Processing of Accurate Evidence Appropriate Selection and Use of Causal Factor Analysis Tools Co-developed Bill Corcoran & Richard Hartley 8-21-07

11. 11 Process Output - Lessons To Be Learned Lessons To Be Learned (LTBL) are: How the Organization Changes the Way We Should Be Thinking and Behaving Developed for Things that Specific Corrective Actions Can’t Solve Confidence in LTBL Requires: Confidence in Factors Knowledge of the Extent of Condition Knowledge of those Additional Factors that Indicate our System is Weak (ECAQ) Confidence in Consequences, Extent, ECAQ Requires: Complete Collection and Processing of Accurate Evidence Appropriate Selection and Use of Causal Factor Analysis Tools Co-developed Bill Corcoran & Richard Hartley 8-21-07

12. 12 Mission Goals Policies Processes Programs Causal Factors Analysis 5 Identify Latent Organizational Weaknesses  HRO Attribute #1 CFA Feedback to HRO Attributes Re-Actively Preventing Accidents – Protect Plant from Worker 3 Identify Human Performance Error Precursors  HRO Attribute #3 Work-as imagined Work-as-done Evaluate Failure to Learn as an Organization 2 Identify Gaps between “work-as-imagined” and “work-as-done”  HRO Attribute #4 Initiating Action 1  “What” Happened Event Facts 4 Identify Flawed Defenses  HRO Attribute #2 Event

13. 13 Where We Are – Where We Are Going General Employee Tier 3  HRO/CFA Concept Introduced to Union Stewards  Training planned for Pantex workers CFA Membership  35 CFA Members trained and experienced  Several investigations conducted  Customer excited about approach & results Tier 2 CFA Leadership  12 CFA team leads trained  9 Seminar Series for Senior Management on HRO – CFA Developed and Tested  Published two HRO & CFA texts to provide theory and practical guide Tier 1