1. Conference on Quality in the Space and Defense Industries - March 3/4, 2008 - Lessons Learned from: The Big Dig Brian Hughitt NASA Headquarters Office of Safety and Mission Assurance

10. Bumper to bumper 6 – 8 hours per day

11. Plus, deteriorating structure

12. Downtown Logan Airport South Boston The solution to the overhead highway…. Downtown Bury it underground

14. The Big Dig – 7 1/2 mile corridor – 161 lane miles – 5 miles of tunnel – 6 interchanges – 200 bridges – 16 million cubic yards of dirt – 541,000 truckloads (4,612 miles of trucks lined up end to end) –15 stadiums filled to the rim with dirt – 3.8 million cubic yards of concrete (enough for a sidewalk from Boston to San Francisco and back 3 times)

15. Accident Synopsis At 11:00 pm on July 10, 2006 a 1991 Buick passenger car occupied by a 46 year-old male driver and his 38 year-old wife was traveling eastbound in the I-90 connector tunnel in Boston, MA, en route to Logan International Airport. As the car approached the end of the connector tunnel, a section of the tunnel’s suspended concrete ceiling (26 tons) detached from the tunnel roof and fell onto the vehicle, crushing its right side. The driver’s wife, occupying the right-front seat, was fatally injured. The driver escaped with minor injuries.

18. National Transportation Safety Board Accident Report Proximate Cause Use of an epoxy anchor adhesive with poor creep resistance Post accident testing revealed that Fast Set epoxy had been supplied and that, while both Fast Set and Standard Set epoxy performed similarly in short term tests, they differed dramatically under long term load. Creep Epoxy is a polymer and its stiffness is time and temperature dependent. If a load is applied suddenly, the epoxy responds like a hard solid. But if that load is then held constant, the molecules within the polymer may begin to rearrange and slide past one another, causing the epoxy to gradually deform. As the deformation increases, it becomes irreversible.

19. Creep: The Unknown Known

20. National Transportation Safety Board Accident Report Contributing Causal Factors Design System Safety Project Management Procurement Supplier Documentation Quality Assurance Industry Standards Governmental Oversight Governmental Regulations

21. Requirements Design Quality Assurance Defenses are never perfect - When Events Lines Up, the Consequences Can Be Devastating Adapted from : James Reason, Managing the Risks of Organizational Accidents, 1997, p. 12 Mishap Manufacturing Test Operations Hazar d

22. Common Quality Threads Product System Process Product Acceptability Hardware Software Materials Design Adequacy Test Inspection & Evaluation Maturity Capability Material Control Process Documentation Process Development Production Realization Process Measurement, Analysis, Improvement And Control Documents Records Procedures Adequacy Availability Control Configuration Management Purchasing Production & Service Metrology Requirements Review Requirement Flow Down Supplier Qualification Maintenance Calibration Process Measurement Internal Audits Process Control Process Analysis Process Improvement Corrective Action Preventative Action Quality System Business System(s) Resource System(s) Config. Mgt. Documentation Commitment Customer Focus Policy Planning Responsibility Review Infrastructure Work Environment Resources Human Resources Workspace Process Equipment Support Services Training Experience Skills Availability Lighting Cleanliness Control Quality Manual Process Maintenance Objectives Integrity Authority Communication Suitability Effectiveness Supplier’s Ability To Provide Suitable Products or Services Design & Development Design Output Design Input Planning Validation Verification Fabrication Control Subtier Control Non-Con Article Control

23. NTSB Accident Report - Contributing Causal Factors - Design Design specifications did not incorporate a provision for attaching a suspended ceiling, even though it was known that one would be needed. Consequently, the tunnel had no embedded ceiling supports. The design consultant repeatedly recommended undercut anchors vs the adhesive anchors ultimately chosen. In order to save costs/time, a change was made to use heavy precast concrete panels in lieu of custom-engineered laminated lightweight concrete panels. “The July 10, 2006, accident was a sudden, violent event, but the circumstances leading up to it developed over a period of more than 20 years, beginning with the design of the Ted Williams tunnel in the late 1980s.”

26. Epoxy Secured Bolts

27. Undercut Anchor System Hole is drilled with an undercut Stud is held in mechanically Anchor Sleeve is inserted and it spreads

28. NTSB Accident Report - Contributing Causal Factors - System Safety No redundancy- the majority of U.S. tunnels have continuous ceiling panels that extend into the concrete wall. If the hangers fail, the ceiling is self-supported. Incomplete Failure Modes and Effects Analysis- Creep not identified as a potential failure mode. Consequently, risk mitigation measures were not implemented.

29. Contributing Causal Factors (cont) Industry Standards ICC AC58: Either a design safety factor of 5.33 or a 120-day creep test is required for Fast Set epoxy. “Given that the ability to sustain a load over a period of time is a typical requirement for almost any type of fastener, the Safety Board is concerned that the ICC has allowed creep testing of epoxy adhesives to be optional. A design engineer should be provided with all of the relevant information about a product before it is used in a safety critical application.”

30. Consequently… - To support product qualification, the supplier provided an Evaluation Report (ER) which included bond strength tables specifying a safety factor of 5.33 for Fast Set epoxy- not the results of creep tests*. -Nothing in the ER tables or footnotes indicated that the Fast Set epoxy should be limited to use with short-term loads regardless of the safety factor employed (a restriction for use in short-term load applications was contained elsewhere in the ER, but could easily have been overlooked). [* The Safety Board learned during the investigation that Fast Set epoxy had been tested for creep performance in 1995 and 1996 and had failed to meet the standard]

31. Contributing Causal Factors (cont) Product Qualification No documentation was provided by the supplier which specified which epoxy formulation was supplied, and neither the contractor nor the design consultant questioned which epoxy was used. Both assumed that the epoxy provided by the supplier was suitable. The Safety Board found fault with the construction contractor and the design consultant for not adequately reviewing the product qualification documentation. But even more….

32. Contributing Causal Factors (cont) “The supplier should have made a clear distinction in all its literature between the relative capabilities of its Standard and Fast Set formulations. It did not do so, even though before the epoxy was provided, the company had conclusive evidence that its Fast Set epoxy was susceptible to creep.” “The Safety Board concludes that the information provided by the supplier was inadequate and misleading.”

33. The Gift… On September 9, 1999, a construction contractor employee installing ventilation ductwork over the tunnel ceiling noticed that several of the anchors had begun to pull out. …and the Smoking Gun On November 12, 1999, a proof load test was performed on one of the anchors that had shown significant displacement (9/16”). The engineer noted that “the bolt held for a few seconds, then began to pull out with almost no resistance”.

35. The Supplier’s Response When the supplier was called to examine the anchor displacements, they seemed surprised that the anchors that had been successfully proof tested only a few months before could be failing. Installation problems (e.g., excessive preload) were postulated as the cause. No evidence was found that the supplier took any follow-up action after the examination. - No further testing - No further research “At least some supplier officials were aware that their Fast Set epoxy was subject to creep, but this information was apparently not considered or was not known by the representatives who evaluated the failed anchors. Even if the information about poor creep resistance was not common knowledge, a reasonable amount of research would likely have revealed it. The Safety Board would have expected the supplier of a safety critical component to have been more proactive in determining why its product was failing.”

36. The Builder, Design Agent, and Project Manager Reply Increased proof load testing

37. The root cause for the hanger displacement was never clearly identified… …and surveillance monitoring inspections were never implemented “The project managers apparently accepted at face value the catalog load capacities provided by the supplier, and performed no independent testing to verify that … the anchors would perform similarly in this particular application.” NTSB Accident Report

38. 20,000 # 15,000 # 10,000 # 5,000 # 0 LbF Design Service Load Post Installation Proof Test Finite Element Analysis 2,600 # Calculated design service load (2,600 Lb-Force)

39. 20,000 # 15,000 # 10,000 # 5,000 # 0 LbF Design Service Load Post Installation Proof Test Finite Element Analysis 2,600 # 3,250 # After each bolt was installed, a proof test was conducted at 25% higher than design service load (3,250 Lb.- Force)

40. 20,000 # 15,000 # 10,000 # 5,000 # 0 LbF Design Service Load Post Installation Proof Test Finite Element Analysis 2,600 # 6,350 # 3,250 # Later, after slippage was noted, bolts were proof tested to the maximum allowable load (6,350 Lb-Force)

41. 15,000 # 10,000 # 5,000 # 0 LbF Design Service Load Post Installation Proof Test Finite Element Analysis 2,600 # 2,371 # 2,823 # 6,350 # 3,250 # A finite element analyses determined that the load would be between 2,371 and 2,823 lb force

42. Where is the voice of Quality? "You’ve noted the key piece of information that is missing. That is the cause of the anchor failure and how the repair procedure will overcome that… We are not trying to hold up construction, we are trying to make a determination that the installation is safe…” Design Manager e-mail concerning response to Deficiency Report “Glaringly absent from the Deficiency Report is any explanation why the anchors failed and what steps are proposed to ensure that this problem does not reoccur.” Structural Engineer e-mail reply

43. The Gift (part 2) On December 17, 2001, a quality control inspector submitted a Noncompliance Report which stated: “Several anchors appear to be pulling away from the concrete. The subject anchors were previously tested to the revised value of 6350 lbs, all of which passed…. Reason for failure is unknown.” “At this point, it should have been obvious... that the remedy that had been developed in response to the anchor displacement in the HOV tunnel in 1999 had not been effective, as anchors that had passed proof testing at higher values were still displacing. This was another opportunity to …inspect all the installed anchors to determine the extent and, more importantly, the cause of the anchor displacement. Instead, the companies apparently considered the continuing failures as isolated instances and took no action to address the problem in a systemic way.” NTSB Accident Report

45. Contributing Causal Factors (cont) Standards and Protocols for the Testing of Adhesive Anchors The Safety Board concluded that proof test procedures are inadequate for adhesive anchors and recommended that, building on current test standards, the FHWA work jointly with industry to develop standards for the testing of adhesive anchors used in sustained tensile-load applications.[ The ICBO AC58 optional creep test was found to be overly simplistic/limited ] ASTM D2990-01, Standard Test Method for Tensile, Compressive, and Flexural Creep and Creep Rupture of Plastics Since the properties of viscoelastic materials are dependent on time…an instantaneous test result cannot be expected to show how a material will behave when subjected to stress or deformation for an extended period of time.

46. Contributing Causal Factors (cont) Lack of Awareness “This accident investigation revealed a striking lack of awareness among designers, contractors, managers, and overseers about the nature and performance of polymer adhesives, even as those adhesives were being approved for use an applications where a failure would present an immediate threat to the public. Even after being presented with evidence of anchor creep, project managers and overseers failed to recognize the inherent weakness in the epoxy adhesive – a weakness that could not be overcome even with the best installation practices or the most rigorous short- term proof testing.”

47. Cognitive Dissonance A psychological term describing the uncomfortable tension that may result from having two conflicting thoughts at the same time, or from engaging in behavior that conflicts with one's beliefs, or from experiencing apparently conflicting phenomena.psychological In simple terms, it can be the filtering of information that conflicts with what you already believe, in an effort to ignore that information and reinforce your beliefs. Social psychologist Leon Festinger first proposed the theory in 1957 after the publication of his book When Prophecy Fails, observing the counterintuitive belief persistence of members of a UFO doomsday cult and their increased proselytization after the leader's prophecy failed.Social psychologistLeon FestingerWhen Prophecy FailsUFO cultproselytization Wikipedia

48. Contributing Causal Factors (cont) Tunnel Inspections In November, 2003, the Design Agent published Inspection Manual for Tunnels and Boat Structures. The manual required each ceiling hanger component to be inspected visually or by NDT. From the time the tunnel was opened to traffic until the day of the fatal accident, no tunnel inspections were performed. Post accident inspection of the suspended ceiling displayed large numbers of anchors that had become displaced (~25%), and that the displacement was so obvious that even a cursory examination would have revealed that structural integrity was threatened.

50. Tunnel Inspections (cont) “Investigators asked MTA officials why the inspection manual was not used... The officials stated that the inspection manual was not used because (1)a tunnel inspection database needed to be developed, (2)the inspection manual was being reviewed by the FHWA and the MTA, and (3)MTA personnel needed time to be trained on the manual. ” NTSB Accident Report

52. Tunnel Inspections (cont) The FHWA National Bridge Inspection Program (NBIP) mandates bridge inspections at least once every two years. There are no similar mandates for tunnel inspections.

53. Contributing Causal Factors (cont) National Standards The FHWA’s Road Tunnel Design Guidelines does not address the design, construction, and inspection of tunnel finishes.

55. “ Quality is free. It is not a gift, but it is free. What costs money are the unquality things- all the actions that involve not doing the job right the first time. — Phillip B. Crosby