Reliability Engineering Applications and Case Studies Mission Success Starts with Safety   Reliability Engineering Applications and Case Studies Fayssal M. Safie, Ph. D., NASA R&M Tech Fellow Marshall Space Flight Center RAM VII Workshop Tutorial Huntsville, Alabama November 4-5, 2013

Agenda Reliability Engineering Overview Reliability Engineering Definitions The Reliability Engineering Case The Relationship to Safety, Mission Success, and Affordability Design VS. Process Reliability Applications and Case Studies The ARES V Case The Roller Bearing Inner Race Fracture Case The Space Shuttle Main Engine (SSME) High Pressure Fuel Turbo-pump (HPFTP) First stage Turbine Blade Case The Space Shuttle Auxiliary Power Unit (APU) Case The Reliability Challenge Backups 2

Reliability Engineering Overview F. Safie

Reliability Engineering Definitions Reliability Engineering is: The application of engineering and scientific principles to the design and processing of products, both hardware and software, for the purpose of meeting product reliability requirements or goals. The ability or capability of the product to perform the specified function in the designated environment for a specified length of time or specified number of cycles Reliability as a Figure of Merit is: The probability that an item will perform its intended function for a specified mission profile.

The Reliability Engineering Case Reliability Program Management & Control Reliability Program Plan Contractors and Suppliers Monitoring Reliability Program Audits Reliability Progress Reports Failure Review Processes Process Reliability Reliability Requirements Root Cause Analysis Design Reliability Drivers Reliability Requirements Analysis Worst Case Analysis Critical Parameter Reliability Requirements Allocation Human Reliability Analysis Process Characterization Reliability Prediction Stress Screening Process Parameter Design Reliability Case Sneak Circuit Analysis Feedback Control Statistical Process Control Probabilistic Design Analysis Process Monitoring Reliability Testing FMEA/CIL

The Relationship to Safety, Mission Success, and Affordability Reliability Maintainability Supportability COST OF LOGISTICS SUPPORT & INFRASTRUCTURE Failure Identification and Analysis Critical Items Identification Design Mitigation and Critical Process Control Level of Repair Spares, Facilities, Maintenance Labor , materials , Maintenance Support , etc. COST OF PREVENTIVE MAINTENANCE Preventive Maintenance AFFORDABILITY COST OF CORRRECTIVE MAINTENANCE Corrective Maintenance Reliability is a critical input for Availability and Affordability. Reliability analysis is critical for understanding component failure mechanisms and integrated system failures; and identifying reliability critical design and process drivers. Reliability analysis and data feeds maintainability and engineering and improves design by identifying critical failures, reducing maintenance manpower needs, reducing lifecycle cost, and provides data essential for project management COST OF LOSS Failures Loss of Crew/Mission/Space System, Stand Down, Loss of Launch Opportunity, etc. COST OF DEVELOPMENT TESTING, CERTIFICATION, AND SUSTAINING ENGINEERING Redesigns

Design VS. Process Reliability “Design it Right and Built it Right” Design Reliability Operational Reliability Process Reliability Process Uniformity Process Control Materials Properties Loads & Environments Operating conditions Design Process Process Capability

Design Reliability 11/8/2018

The Challenger Accident Design Reliability The Challenger Accident

The Challenger Accident Design Reliability The Challenger Accident Causes and Contributing Factors The zinc chromate putty frequently failed and permitted the gas to erode the primary O-rings. The particular material used in the manufacture of the shuttle O-rings was the wrong material to use at low temperatures. Elastomers become brittle at low temperatures. This is a schematic of a SRM field joint identifying the leak path of the combustion gas and how it would escape to the outside. Zinc chromate putty, added between the joint segments to protect the O-rings from high temperature and high pressure gases, frequently failed.

The Challenger Accident Design Reliability The Challenger Accident

The Challenger Accident Case Concluding Remarks The clear message from the Challenger accident case is that understanding design reliability is critical to the overall system reliability and safety. This includes: Understanding failure mechanisms Understanding the loads and environment Understanding the material capabilities Operating within the design environment Physics based reliability analysis is critical to understand failure mechanism and design uncertainties

Process Reliability 11/8/2018

Process Reliability The Columbia Accident Causes and Contributing Factors Breach in the Thermal Protection System caused by the left bipod ramp insulation foam striking the left wing leading edge. There were large gaps in NASA's knowledge about the foam. cryopumping and cryoingestion, were experienced during tanking, launch, and ascent. Dissections of foam revealed subsurface flaws and defects as contributing to the loss of foam. In summary, we can trace the root cause of the Columbia accident to Physical and organizational causes. The physical cause was a breach in the Thermal Protection System caused by the left bipod ramp insulation foam striking the left wing leading edge. Organizational Root Causes include:  Compromises that were required to gain approval for the Shuttle Subsequent years of resource constraints Fluctuating priorities Schedule pressures Mischaracterization of the Shuttle as operational rather than developmental Lack of an agreed national vision for human space flight Reliance on past success as a substitute for sound engineering practices Organizational barriers that prevented effective communication of critical safety information and stifled professional differences of opinion Lack of integrated management across program elements, and The evolution of an informal chain of command and decision-making processes that operated outside the organization's rules.

Process Reliability The Columbia Accident The ET thermal protection system is a foam-type material applied to the external tank to maintain cryogenic propellant quality, minimize ice and frost formation, and protect the structure from ascent, plume, and re-entry heating. The TPS during re-entry is needed because after ET/Orbiter separation, premature structural overheating due to loss of TPS could result in a premature ET breakup with debris landing outside the predicted footprint.

Process Reliability The Columbia Accident The ET TPS Reliability The reliability of the TPS is broadly defined as its strength versus the stress put on it in flight. High TPS reliability means less debris released and fewer hits to the orbiter, reducing system risk. Process control, process uniformity, high process capability are critical factors in achieving high TPS reliability. Good process uniformity and high process capability yield fewer process defects, smaller defect sizes, and good material properties that meets the engineering specification—the critical ingredients of high reliability. 11/8/2018

Process Reliability The Columbia Accident Foam Spray Process Evaluation Process variability was evaluated after the fact Dissection data collected after the Columbia accident showed excessive variability (Coefficient of variation is greater than 100%) Within tank variability was high, and tank to tank variability could not be fully characterized Defect/void characterization was difficult and statistics derived had high level of uncertainty The natural variation of the process was not well understood The relationship between process variables and defects was not known F. Safie

Process Reliability The Columbia Accident Process Enhanced Foam Conducted verification and validation testing sufficient enough to understand and characterize the process variability and process capability Evaluated process uniformity Evaluated process capability for meeting the specification Statistical evaluation of the data showed that significant improvements were made in process uniformity and process capability, including significant reduction in the coefficient of variation (COV) of the process critical output parameters (e.g. void frequency and void sizes) F. Safie

The Quality, Reliability, and Risk Relationship Process Reliability The Process Reliability Impact The Quality, Reliability, and Risk Relationship Process Reliability Component Reliability System Risk High Process Uniformity and Process Capability Capability vs. Performance Failure Impact on System High Material Capability Higher Reliability Lower Risk and Higher Safety

Foam Probabilistic Risk Assessment Input Data Validation Data ET TPS Dissections (ET Project) TPS Void Distributions Process Control ET Dissection / Manufacturing Data TPS Debris Generation (divot/no divot, size/shape, (mass), time and location of release, and pop-off velocity TPS Reliability TPS Geometry Properties, Boundary Conditions (ET Project) Thermal-Vacuum and Flight Imagery Data Debris Transport and CFD Calculations (SE&I) TPS Transport Model (axial/lateral locations and velocities during ascent Debris Transport Analysis Orbiter Geometric Models (Orbiter Project) Orbiter Impact Algorithms (impact/no impact, location, time, mass, velocity and angle) Orbiter Post-Flight Data Orbiter Impact / Damage Tolerances (Orbiter Project) Orbiter Damage Analysis (tile/RCC panel damage) Probability of Orbiter Damage Exceeding Damage Tolerance System Risk 11/8/2018

The Columbia Accident Case Concluding Remarks The clear messages from the Columbia accident are: Integrated failure analysis is critical to understand the relationship between component reliability, and system safety. Inadequate manufacturing and quality control can have a severe negative impact on component reliability and system safety. Process design should be considered upfront in the overall design process.

Reliability Applications and Case Studies F. Safie

The ARES V Case F. Safie

Trades During Conceptual Phase Notional

Reliability Methodology The Process Vehicle Configuration Subsystem Parameters Mission Profile Advanced Concepts Office Design Input Reliability Database Reliability Algorithms Ares V Subsystem Data Mission Performance Data System Analysis Integration Reliability Evaluation Results Event Time (sec)   Alt (km) Liftoff Maximum Q SRB Separation Shroud Separation Main Engine Cutoff EDS Ignition EDS Engine Cutoff 25

Reliability Methodology - Notional The Input / Output Strap-On Core EDS System-Level Results No. of engines? Engine type? Burntime? Power Level? No. of segments? Propellant type? Mission Profile Vehicle Results EDS Core Reliability Data Strap-On EDS Air-Start EDS Non-Cat EDS Cat EDS Other Core Non-Cat Core Cat Core Other Configuration SRB RSRM Strap-On Separation

Launch Vehicle Comparison Example 360.5' 71.1' 179.2' 215.6' 73.8' 33.0' Notional Booster Stage (each) 2 / 5 - Segment SRM First Stage 5 / SSME Blk II @ 104.5% Second Stage 2 / J-2S+ @ 100% Booster Stage (each) 2 / 5 - Segment SRM First Stage 5 / RS-68 Second Stage 1 / J-2X

Earth Departure Stage (EDS) Within Concept Trade Earth Departure Stage (EDS) Performance-based reliability analysis provided supporting data in key architecture, element, subsystem, and component design decisions. Earth Departure Stage Solar Array to Fuel Cells

Notional Mission Reliability Over the Mission Profile Liftoff thru Core Stage MECO LOM = 1/107 Core Stage MECO thru Orbit Insertion LOM = 1/590 Orbit Insertion thru TLI LOM = 1/210 Core Stage Separation & EDS J-2X Ignition EDS Engine Cutoff LSAM/CEV Separation Notional Shroud Separation EDS TLI Burn SRB Separation Liftoff EDS Disposal Launch Site SRB Splashdown Core Stage Impact CEV Rendezvous & Dock w/ EDS Core Stage MECO thru TLI LOM = 1/153 Liftoff thru Orbit Insertion LOM = 1/87 Liftoff thru TLI LOM = 1/62

The ARES V Case Concluding Remarks Reliability is a critical system parameter that needs to be considered upfront in the design process along with performance and cost. Adopting a “Design for Reliability” philosophy is key in achieving ambitious goals in safety and affordability. Reliability trade studies are part of a risk informed process to support architecture capability studies and conceptual design trades.

The Roller Bearing Inner Race Fracture Case

Background During rig testing the AT/HPFTP Bearing experienced several cracked races. Three of four tests failed (440C bearing races Fractured) 11/8/2018

Objective In this application, an analysis was done for the Pratt & Whitney Alternate Turbo-pump Development (ATD) to assist in a High Pressure Fuel Turbo-pump (HPFTP) roller bearing inner race fracture problem. In particular, the questions which needed to be addressed were: The probability of failure due to the hoop stress exceeding the materials capa­bility strength was acceptable. The effect of manufacturing stresses on the fracture probability. There were two different materials under consideration; the 440C (current material) and the 9310. 11/8/2018

Probabilistic Engineering Analysis Probabilistic engineering analysis is used when data is limited and the design is characterized by complex geometry or is sensitive to loads, material properties, and environments. Failure Region 11/8/2018

The Analytical Approach The Simulation Model 11/8/2018

The Simulation Model Since this failure model is a simple overstress model, only two distributions need to be simulated: the hoop stress distribution and the materials capability distribution. In order to calculate the hoop stress distribution it was necessary to determine the materials properties variability. Of those materials properties that af­fected the total inner race hoop stress, a series of equations was derived which mapped these life drivers (such as modulus of elasticity, coefficient of thermal expansion, etc.) into the total Inner race hoop stress. In order to derive these equations, several sources of information were used which included P&W computer "design programs, equations from engineering theory, manufacturing stress data, and engineering judgment. This resulted in a distribution of the total hoop stress.

The Simulation Model In a similar fashion, a distribution on the materials ca­pability strength was derived. In this case, life drivers such as fracture toughness, crack depth/length, yield strength. etc. were important. The resulting materials capability strength distribution was then obtained through a similar series of equations. The Monte Carlo simulation in this case would calculate a random hoop stress and a random materials capability strength. if the former is greater than the later, a failure due to overstress occurs in the simulation. Otherwise, a success is recorded. The simulation was run for two different materials: 440C (current material) and 9310. After several thousand simulations are conducted, the percent which failed are recorded.

440C w/ actual* mfg. stresses The Analysis Results Test Failures Race Configuration Failures in 100,000 firings** 3 of 4 440C w/ actual* mfg. stresses 68,000 N/A 440C w /no mfg. stresses 1,500 440 C w/ ideal mfg. stresses 27,000 0 of 15 9310 w/ ideal mfg. stresses 10 The results of this analysis clearly show that the 9310 material was preferred over the 440C in terms inner race fracture failure mode. *ideal + abusive grinding **Probabilistic Structural Analysis 11/8/2018

The Roller Bearing Inner Race Fracture Case Concluding Remarks The results of this analysis clearly showed that the 9310 material was preferred over the 440C in terms of the inner race fracture failure mode. Manufacturing stresses effect for the 440C material was very significant. Material selection has a major impact on Reliability. Probabilistic engineering analysis is critical to perform sensitivity analysis and trade studies for material selection and testing. 11/8/2018

The Space Shuttle Main Engine High Pressure Fuel Turbo-pump (HPFTP) First stage Turbine Blade Case

HPFTP First Stage Turbine Blade Cracks Objective Determine the Space Shuttle flight risk due to a HPFTP first stage turbine blade failure HPFTP

Background A crack was found in a first stage turbine blade in HPFTP development unit 2423 during dye penetrant inspection 1/19/96 (Firtree Lobe Crack) The subject blade had accumulated 20 starts and 9,826 seconds of operation. A total of 34 blade set of the current configuration have been dye penetrant inspected, with no other crack being found (see Database: Case 1). Metallurgical evaluation of blade: Fracture is hydrogen assisted cracking Fracture origin approximately in middle of bottom firtree lobe- starting on pressure side No clear evidence of crack progression (striations) Etc.

Assumptions A crack in a blade is a failure Only last dye penetrant inspection times are used (34 sets) One failure (crack) at 20 starts and 9826 seconds

Database

Analysis Results The starts and run time for the three pumps: 2 STARTS/817 SEC 2 STARTS/780 SEC 4 STARTS /1856 SEC Weibull model was used for reliability predictions

The Roller Bearing Inner Race Fracture Case Concluding Remarks Manufacturing records review for the flight set showed no discrepancies Fleet leader blade set with 22241 seconds and 46 tests 53 blade sets tested greater than the flight units. Flight Reliability was assessed and risk was accepted by Shuttle program.

The Space Shuttle Auxiliary Power Unit (APU) Case

Objective Post Challenger Accident, a major simulation modeling effort was conducted to evaluate the reliability of the Shuttle APU turbine wheel. Specifically, The simulation model was designed to determine the probability of failure of the APU turbine wheel due to a critical blade crack given that the wheel has to operate for some specified life limit during which a given inspection policy is imposed. The simulation model was also designed to allow the analyst to study the trade-offs between wheel reliability, wheel life, inspection interval, and rejection crack size. 11/8/2018

APU Turbine Blade Cracks Background APU Turbine Blade Cracks RIM

Basic Approach 11/8/2018

Data Screening and Classification The data used in this analysis were taken from all of the crack data that was available for the turbine wheel blades. Several steps were taken to edit the cumulative data into a form which could be used in the analysis. The first simplification in the analysis was to consider just the root cracks (not tip cracks). This simplification was justified because it has been stated that only root cracks can cause the loss of a blade or the failure of a wheel. Root cracks are subdivided into Fillet and outboard cracks. For the purposes of fitting crack growth models to the data, only data for cracks with three or more mappings were considered.

Crack Initiation Model The first modeling effort needed in order to describe the crack growth in the blades was modeling the probability of crack initiation. Because two types of cracks were considered, the probability of crack initiation was required for each type. For simplicity, it was assumed that the two types of cracks initiate independently of each other and that cracks on different blades of a wheel initiate independently of each other.

Crack Propagation Model Where, t is cumulative time; and a and b are the growth parameters

The Simulation Flowchart

The Analysis Using the simulation model, an analysis was conducted to determine the APU turbine wheel reliability for a predetermined inspection policy. The inspection policy considered consisted of 16 HGS inspection interval, 100 HGS wheel life limit, and 0.090 in rejection flaw size. Using this inspection policy, a simulation based analysis was performed, and a wheel reliability of 0.99994 was obtained.

The Space Shuttle Auxiliary Power Unit (APU) Case Concluding Remarks A simulation model was developed which allowed the Shuttle program to determine the probability of failure of the APU turbine wheel due to a critical blade crack, given that the wheel has to operate for some specified life limit during which a given inspection policy is imposed. In addition, the simulation model allows the analyst to study the trade-offs between reliability, wheel life, inspection interval, and rejection crack size. The inspection policy derived by the simulation model was used by the Shuttle program through out it’s operational phase to maintain a reliable and safe APU turbine wheel fleet. 11/8/2018

Backups F. Safie

Reliability Challenges Embedding reliability engineers in the design engineering community to effectively help the design process. Training our engineering community to have a better understanding of the language of probability, statistics, and reliability engineering. Integrating reliability, maintainability, and supportability (RMS) analyses, a key to reduce sustainment cost and achieve high system safety and availability.