1. Reliability Engineering Applications and Case Studies
Mission Success Starts with Safety
Reliability Engineering
Applications and Case Studies
Fayssal M. Safie, Ph. D.,
NASA R&M Tech Fellow
Marshall Space Flight Center
RAM VII Workshop Tutorial
Huntsville, Alabama
November 4-5, 2013

2. Agenda Reliability Engineering Overview
Reliability Engineering Definitions
The Reliability Engineering Case
The Relationship to Safety, Mission Success, and Affordability
Design VS. Process Reliability
Applications and Case Studies
The ARES V Case
The Roller Bearing Inner Race Fracture Case
The Space Shuttle Main Engine (SSME) High Pressure Fuel Turbo-pump (HPFTP) First stage Turbine Blade Case
The Space Shuttle Auxiliary Power Unit (APU) Case
The Reliability Challenge
Backups 2

3. Reliability Engineering Overview
F. Safie

4. Reliability Engineering Definitions
Reliability Engineering is:
The application of engineering and scientific principles to the design and processing of products, both hardware and software, for the purpose of meeting product reliability requirements or goals.
The ability or capability of the product to perform the specified function in the designated environment for a specified length of time or specified number of cycles
Reliability as a Figure of Merit is: The probability that an item will perform its intended function for a specified mission profile.

5. The Reliability Engineering Case
Reliability Program Management & Control
Reliability Program Plan
Contractors and Suppliers Monitoring
Reliability Program Audits
Reliability Progress Reports
Failure Review Processes
Process Reliability
Reliability Requirements
Root Cause Analysis
Design Reliability Drivers
Reliability Requirements Analysis
Worst Case Analysis
Critical Parameter
Reliability Requirements Allocation
Human Reliability
Analysis
Process Characterization
Reliability Prediction
Stress Screening
Process Parameter Design
Reliability
Case
Sneak Circuit Analysis
Feedback Control
Statistical Process Control
Probabilistic Design Analysis
Process Monitoring
Reliability Testing
FMEA/CIL

6. The Relationship to Safety, Mission Success, and Affordability
Reliability
Maintainability
Supportability
COST OF LOGISTICS SUPPORT & INFRASTRUCTURE
Failure
Identification and Analysis
Critical Items
Identification
Design Mitigation and Critical Process Control
Level of Repair
Spares, Facilities,
Maintenance Labor , materials , Maintenance Support , etc.
COST OF PREVENTIVE MAINTENANCE
Preventive Maintenance
AFFORDABILITY
COST OF CORRRECTIVE MAINTENANCE
Corrective Maintenance
Reliability is a critical input for Availability and Affordability. Reliability analysis is critical for understanding component failure mechanisms and integrated system failures; and identifying reliability critical design and process drivers.
Reliability analysis and data feeds maintainability and engineering and improves design by identifying critical failures, reducing maintenance manpower needs, reducing lifecycle cost, and provides data essential for project management
COST OF LOSS
Failures
Loss of Crew/Mission/Space System, Stand Down, Loss of Launch Opportunity, etc.
COST OF DEVELOPMENT TESTING, CERTIFICATION, AND SUSTAINING ENGINEERING
Redesigns

7. Design VS. Process Reliability “Design it Right and Built it Right”
Design Reliability
Operational
Reliability
Process Reliability
Process Uniformity
Process Control
Materials Properties
Loads &
Environments
Operating conditions
Design
Process
Process
Capability

8. Design Reliability
11/8/2018

9. The Challenger Accident
Design Reliability
The Challenger Accident

10. The Challenger Accident
Design Reliability
The Challenger Accident
Causes and Contributing Factors
The zinc chromate putty frequently failed and permitted the gas to erode the primary O-rings.
The particular material used in the manufacture of the shuttle O-rings was the wrong material to use at low temperatures.
Elastomers become brittle at low temperatures.
This is a schematic of a SRM field joint identifying the leak path of the combustion gas and how it would escape to the outside. Zinc chromate putty, added between the joint segments to protect the O-rings from high temperature and high pressure gases, frequently failed.

11. The Challenger Accident
Design Reliability
The Challenger Accident

12. The Challenger Accident Case
Concluding Remarks
The clear message from the Challenger accident case is that understanding design reliability is critical to the overall system reliability and safety. This includes:
Understanding failure mechanisms
Understanding the loads and environment
Understanding the material capabilities
Operating within the design environment
Physics based reliability analysis is critical to understand failure mechanism and design uncertainties

13. Process Reliability
11/8/2018

14. Process Reliability The Columbia Accident
Causes and Contributing Factors
Breach in the Thermal Protection System caused by the left bipod ramp insulation foam striking the left wing leading edge.
There were large gaps in NASA's knowledge about the foam.
cryopumping and cryoingestion, were experienced during tanking, launch, and ascent.
Dissections of foam revealed subsurface flaws and defects as contributing to the loss of foam.
In summary, we can trace the root cause of the Columbia accident to Physical and organizational causes. The physical cause was a breach in the Thermal Protection System caused by the left bipod ramp insulation foam striking the left wing leading edge.
Organizational Root Causes include: 
Compromises that were required to gain approval for the Shuttle
Subsequent years of resource constraints
Fluctuating priorities
Schedule pressures
Mischaracterization of the Shuttle as operational rather than developmental
Lack of an agreed national vision for human space flight
Reliance on past success as a substitute for sound engineering practices
Organizational barriers that prevented effective communication of critical safety information and stifled professional differences of opinion
Lack of integrated management across program elements, and
The evolution of an informal chain of command and decision-making processes that operated outside the organization's rules.

15. Process Reliability The Columbia Accident
The ET thermal protection system is a foam-type material applied to the external tank to maintain cryogenic propellant quality, minimize ice and frost formation, and protect the structure from ascent, plume, and re-entry heating.
The TPS during re-entry is needed because after ET/Orbiter separation, premature structural overheating due to loss of TPS could result in a premature ET breakup with debris landing outside the predicted footprint.

16. Process Reliability The Columbia Accident
The ET TPS Reliability
The reliability of the TPS is broadly defined as its strength versus the stress put on it in flight.
High TPS reliability means less debris released and fewer hits to the orbiter, reducing system risk.
Process control, process uniformity, high process capability are critical factors in achieving high TPS reliability.
Good process uniformity and high process capability yield fewer process defects, smaller defect sizes, and good material properties that meets the engineering specification—the critical ingredients of high reliability.
11/8/2018

17. Process Reliability The Columbia Accident
Foam Spray Process Evaluation
Process variability was evaluated after the fact
Dissection data collected after the Columbia accident showed excessive variability (Coefficient of variation is greater than 100%)
Within tank variability was high, and tank to tank variability could not be fully characterized
Defect/void characterization was difficult and statistics derived had high level of uncertainty
The natural variation of the process was not well understood
The relationship between process variables and defects was not known
F. Safie

18. Process Reliability The Columbia Accident
Process Enhanced Foam
Conducted verification and validation testing sufficient enough to understand and characterize the process variability and process capability
Evaluated process uniformity
Evaluated process capability for meeting the specification
Statistical evaluation of the data showed that significant improvements were made in process uniformity and process capability, including significant reduction in the coefficient of variation (COV) of the process critical output parameters (e.g. void frequency and void sizes)
F. Safie

19. The Quality, Reliability, and Risk Relationship
Process Reliability
The Process Reliability Impact
The Quality, Reliability, and Risk Relationship
Process Reliability
Component Reliability
System Risk
High Process Uniformity and Process Capability
Capability vs. Performance
Failure Impact on System
High Material Capability
Higher Reliability
Lower Risk and Higher Safety

20. Foam Probabilistic Risk Assessment
Input Data
Validation Data
ET TPS Dissections
(ET Project)
TPS Void Distributions
Process Control
ET Dissection / Manufacturing Data
TPS Debris Generation (divot/no divot, size/shape, (mass), time and location of release, and pop-off velocity
TPS Reliability
TPS Geometry Properties, Boundary Conditions
(ET Project)
Thermal-Vacuum and Flight Imagery Data
Debris Transport and CFD Calculations
(SE&I)
TPS Transport Model (axial/lateral locations and velocities during ascent
Debris Transport Analysis
Orbiter Geometric Models
(Orbiter Project)
Orbiter Impact Algorithms (impact/no impact, location, time, mass, velocity and angle)
Orbiter Post-Flight Data
Orbiter Impact / Damage Tolerances
(Orbiter Project)
Orbiter Damage Analysis (tile/RCC panel damage)
Probability of Orbiter Damage Exceeding Damage Tolerance
System Risk
11/8/2018

21. The Columbia Accident Case
Concluding Remarks
The clear messages from the Columbia accident are:
Integrated failure analysis is critical to understand the relationship between component reliability, and system safety.
Inadequate manufacturing and quality control can have a severe negative impact on component reliability and system safety.
Process design should be considered upfront in the overall design process.

22. Reliability Applications and Case Studies
F. Safie

23. The ARES V Case
F. Safie

24. Trades During Conceptual Phase
Notional

25. Reliability Methodology The Process
Vehicle Configuration
Subsystem Parameters
Mission Profile
Advanced Concepts Office Design Input
Reliability Database
Reliability Algorithms
Ares V Subsystem Data
Mission Performance
Data
System Analysis Integration
Reliability Evaluation Results
Event
Time (sec)
Alt (km)
Liftoff
Maximum Q
SRB Separation
Shroud Separation
Main Engine Cutoff
EDS Ignition
EDS Engine Cutoff 25

26. Reliability Methodology - Notional The Input / Output
Strap-On
Core
EDS
System-Level Results
No. of engines?
Engine type?
Burntime?
Power Level?
No. of segments?
Propellant type?
Mission Profile
Vehicle Results
EDS
Core
Reliability Data
Strap-On
EDS Air-Start
EDS Non-Cat
EDS Cat
EDS Other
Core Non-Cat
Core Cat
Core Other
Configuration
SRB
RSRM
Strap-On Separation

27. Launch Vehicle Comparison Example
360.5'
71.1'
179.2'
215.6'
73.8'
33.0'
Notional
Booster Stage (each)
2 / 5 - Segment SRM
First Stage
5 / SSME Blk 104.5%
Second Stage
2 / 100%
Booster Stage (each)
2 / 5 - Segment SRM
First Stage
5 / RS-68
Second Stage
1 / J-2X

28. Earth Departure Stage (EDS)
Within Concept Trade
Earth Departure Stage (EDS)
Performance-based reliability analysis provided supporting data in key architecture, element, subsystem, and component design decisions.
Earth Departure Stage
Solar Array to Fuel Cells

29. Notional Mission Reliability Over the Mission Profile
Liftoff thru Core Stage MECO LOM = 1/107
Core Stage MECO thru Orbit Insertion LOM = 1/590
Orbit Insertion thru TLI LOM = 1/210
Core Stage Separation & EDS J-2X Ignition
EDS Engine Cutoff
LSAM/CEV
Separation
Notional
Shroud Separation
EDS TLI Burn
SRB Separation
Liftoff
EDS Disposal
Launch
Site
SRB
Splashdown
Core Stage
Impact
CEV Rendezvous &
Dock w/ EDS
Core Stage MECO thru TLI LOM = 1/153
Liftoff thru Orbit Insertion LOM = 1/87
Liftoff thru TLI LOM = 1/62

30. The ARES V Case Concluding Remarks
Reliability is a critical system parameter that needs to be considered upfront in the design process along with performance and cost.
Adopting a “Design for Reliability” philosophy is key in achieving ambitious goals in safety and affordability.
Reliability trade studies are part of a risk informed process to support architecture capability studies and conceptual design trades.

31. The Roller Bearing Inner Race Fracture Case

32. Background
During rig testing the AT/HPFTP Bearing experienced several cracked races. Three of four tests failed (440C bearing races Fractured)
11/8/2018

33. Objective
In this application, an analysis was done for the Pratt & Whitney Alternate Turbo-pump Development (ATD) to assist in a High Pressure Fuel Turbo-pump (HPFTP) roller bearing inner race fracture problem.
In particular, the questions which needed to be addressed were:
The probability of failure due to the hoop stress exceeding the materials capa­bility strength was acceptable.
The effect of manufacturing stresses on the fracture probability.
There were two different materials under consideration; the 440C (current material) and the 9310.
11/8/2018

34. Probabilistic Engineering Analysis
Probabilistic engineering analysis is used when data is limited and the design is characterized by complex geometry or is sensitive to loads, material properties, and environments.
Failure Region
11/8/2018

35. The Analytical Approach The Simulation Model
11/8/2018

36. The Simulation Model
Since this failure model is a simple overstress model, only two distributions need to be simulated: the hoop stress distribution and the materials capability distribution.
In order to calculate the hoop stress distribution it was necessary to determine the materials properties variability.
Of those materials properties that af­fected the total inner race hoop stress, a series of equations was derived which mapped these life drivers (such as modulus of elasticity, coefficient of thermal expansion, etc.) into the total Inner race hoop stress.
In order to derive these equations, several sources of information were used which included P&W computer "design programs, equations from engineering theory, manufacturing stress data, and engineering judgment. This resulted in a distribution of the total hoop stress.

37. The Simulation Model
In a similar fashion, a distribution on the materials ca­pability strength was derived.
In this case, life drivers such as fracture toughness, crack depth/length, yield strength. etc. were important. The resulting materials capability strength distribution was then obtained through a similar series of equations.
The Monte Carlo simulation in this case would calculate a random hoop stress and a random materials capability strength. if the former is greater than the later, a failure due to overstress occurs in the simulation. Otherwise, a success is recorded.
The simulation was run for two different materials: 440C (current material) and 9310.
After several thousand simulations are conducted, the percent which failed are recorded.

38. 440C w/ actual* mfg. stresses
The Analysis Results
Test Failures
Race Configuration
Failures in 100,000 firings**
3 of 4
440C w/ actual* mfg. stresses
68,000
N/A
440C w /no mfg. stresses
1,500
440 C w/ ideal mfg. stresses
27,000
0 of 15
9310 w/ ideal mfg. stresses 10
The results of this analysis clearly show that the 9310 material was preferred over the 440C in terms inner race fracture failure mode.
*ideal + abusive grinding
**Probabilistic Structural Analysis
11/8/2018

39. The Roller Bearing Inner Race Fracture Case Concluding Remarks
The results of this analysis clearly showed that the 9310 material was preferred over the 440C in terms of the inner race fracture failure mode.
Manufacturing stresses effect for the 440C material was very significant.
Material selection has a major impact on Reliability.
Probabilistic engineering analysis is critical to perform sensitivity analysis and trade studies for material selection and testing.
11/8/2018

40. The Space Shuttle Main Engine High Pressure Fuel Turbo-pump (HPFTP) First stage Turbine Blade Case

41. HPFTP First Stage Turbine Blade Cracks
Objective
Determine the Space Shuttle flight risk due to a HPFTP first stage turbine blade failure
HPFTP

42. Background
A crack was found in a first stage turbine blade in HPFTP development unit 2423 during dye penetrant inspection 1/19/96 (Firtree Lobe Crack)
The subject blade had accumulated 20 starts and 9,826 seconds of operation.
A total of 34 blade set of the current configuration have been dye penetrant inspected, with no other crack being found (see Database: Case 1).
Metallurgical evaluation of blade:
Fracture is hydrogen assisted cracking
Fracture origin approximately in middle of bottom firtree lobe- starting on pressure side
No clear evidence of crack progression (striations)
Etc.

43. Assumptions A crack in a blade is a failure
Only last dye penetrant inspection times are used (34 sets)
One failure (crack) at 20 starts and 9826 seconds

44. Database

45. Analysis Results The starts and run time for the three pumps:
2 STARTS/817 SEC
2 STARTS/780 SEC
4 STARTS /1856 SEC
Weibull model was used for reliability predictions

46. The Roller Bearing Inner Race Fracture Case Concluding Remarks
Manufacturing records review for the flight set showed no discrepancies
Fleet leader blade set with seconds and 46 tests
53 blade sets tested greater than the flight units.
Flight Reliability was assessed and risk was accepted by Shuttle program.

47. The Space Shuttle Auxiliary Power
Unit (APU) Case

48. Objective
Post Challenger Accident, a major simulation modeling effort was conducted to evaluate the reliability of the Shuttle APU turbine wheel. Specifically,
The simulation model was designed to determine the probability of failure of the APU turbine wheel due to a critical blade crack given that the wheel has to operate for some specified life limit during which a given inspection policy is imposed.
The simulation model was also designed to allow the analyst to study the trade-offs between wheel reliability, wheel life, inspection interval, and rejection crack size.
11/8/2018

49. APU Turbine Blade Cracks
Background
APU Turbine Blade Cracks
RIM

50. Basic Approach
11/8/2018

51. Data Screening and Classification
The data used in this analysis were taken from all of the crack data that was available for the turbine wheel blades.
Several steps were taken to edit the cumulative data into a form which could be used in the analysis.
The first simplification in the analysis was to consider just the root cracks (not tip cracks). This simplification was justified because it has been stated that only root cracks can cause the loss of a blade or the failure of a wheel.
Root cracks are subdivided into Fillet and outboard cracks.
For the purposes of fitting crack growth models to the data, only data for cracks with three or more mappings were considered.

52. Crack Initiation Model
The first modeling effort needed in order to describe the crack growth in the blades was modeling the probability of crack initiation.
Because two types of cracks were considered, the probability of crack initiation was required for each type.
For simplicity, it was assumed that the two types of cracks initiate independently of each other and that cracks on different blades of a wheel initiate independently of each other.

53. Crack Propagation Model
Where, t is cumulative time;
and a and b are the growth
parameters

54. The Simulation Flowchart

55. The Analysis
Using the simulation model, an analysis was conducted to determine the APU turbine wheel reliability for a predetermined inspection policy.
The inspection policy considered consisted of 16 HGS inspection interval, 100 HGS wheel life limit, and in rejection flaw size.
Using this inspection policy, a simulation based analysis was performed, and a wheel reliability of was obtained.

56. The Space Shuttle Auxiliary Power Unit (APU) Case Concluding Remarks
A simulation model was developed which allowed the Shuttle program to determine the probability of failure of the APU turbine wheel due to a critical blade crack, given that the wheel has to operate for some specified life limit during which a given inspection policy is imposed.
In addition, the simulation model allows the analyst to study the trade-offs between reliability, wheel life, inspection interval, and rejection crack size.
The inspection policy derived by the simulation model was used by the Shuttle program through out it’s operational phase to maintain a reliable and safe APU turbine wheel fleet.
11/8/2018

57. Backups
F. Safie

58. Reliability Challenges
Embedding reliability engineers in the design engineering community to effectively help the design process.
Training our engineering community to have a better understanding of the language of probability, statistics, and reliability engineering.
Integrating reliability, maintainability, and supportability (RMS) analyses, a key to reduce sustainment cost and achieve high system safety and availability.