National Aeronautics and Space Administration From Determinism to “Probabilism” Changing our mindsets, or why PTC isn’t an easy sell - yet

Deterministic DesignProbabilistic Design ? Safety Factors PD DFMR SDF SWP PDF PRA DOE Predictive Models & Algorithms Diagnostics & Prognostics RAM Design Redundancy Life Cycle Prediction FMECA, FTA, Block Diagrams, Systems of Systems Models My Muddled Mind Robust Design

System Safety Tools Feed Risk Informed Decision Making Bryan O’Connor, Chief, Safety and Mission Assurance, NASA SEPT 8, 2007, Huntsville, AL

Using the Ares quantitative safety and reliability requirement to enforce the “design for reliability and safety” paradigm shift Improving system safety by using a functional analysis system approach to model and understand integrated system failures similar to the Shuttle foam problem. Improving design reliability by using probabilistic engineering physics-based modeling Evaluating and understating design uncertainty and design margins using probabilistic engineering techniques Excerpted from Chris Cianciola, S&MA, MSFC, CQSDI March 2008 S&MA in Ares Design – Summary

Uncertainties sources manufacturing, storage, aging, use environments, and scenarios Uncertainties types randomness, lack of knowledge Model to predict uncertainties in systems Uncertainties outside acceptable limits? Uncertainties produce inadequate margins? Most important contributors to uncertainty (sensitivity analyses)? Physics model form uncertainty Sensitivity Analysis Only several contribute most of the uncertainty in system response Requirements Uncertainty Uncertainty Quantification and Risk Assessment risk-informed decision making

One of many decision variables Subjective information - scientific and engineering judgment is necessary Known unknowns and unknown unknowns exist Resources are limited Results uncertainty and credibility – challenges decision makers Social, economic, and political factors exist Probabilistic Models Limitations

Is the methods framework credible? Address the credibility concern with verification and validation Credibility

Start with robustness and margin in design A 30% design margin is not plausible for space flight – it’s just too expensive Our margins are so thin that we really need to investigate margin In our case we want to whittle the margin down as much as possible but still maintain the robustness we need Getting those design margin trades into the system analysis is critical NASA special study data 2006 Special study data

We still use deterministic design and build margin into designs even though we have the computational capabilities to us PT methods We do not have probabilistic design methods in place Probabilistic methods will have to be an investment by NASA into the long term technology base and tools First you need a proper deterministic goal and then you can successfully apply a probabilistic model Numerical Propulsion System Simulation is something that must be implemented to get this country to Mars The entire thought process needs to change, this is not a deterministic world – everything is probabilistic In the deterministic method there is no role for uncertainty or margin Right now technology is so fast we are dumping technology on people who are not educated to use these technologies appropriately NASA special study data 2006 Special study data

Redundancy is a part of our culture right now because it is much easier than looking for a solution using margin When you consider uncertainty in the design phase you are able to examine the trade space more efficiently and thoroughly. The nature of the probabilistic skills dealing in design is very different from the traditional PRA. NASA special study data 2006 Special study data

Design engineers don’t understand how to account for uncertainty in the design process and how to explore the trades throughout In the past we’ve had so much money and time for testing, but we don’t have this anymore Apollo did 14,000 tests on the LEM, do you think we could do that now? We must trade ability to test versus desired reliability demonstration Why PTC Now? NASA special study data 2006

Probabilistic requirements ultimately relate to risk in achieving a level of performance or safety. The most important reason for using a structured approach is to ensure that the potential failure modes of a design are identified early in the design process to better understand the risks. An understanding of risks due to incomplete or inaccurate modeling of a design and the operational environment, or due to uncertainty related to the data used in a model is necessary so that a requirement can be verified with confidence. Surendra N. Singhal, Engineering Directorate, Marshall Space Flight Center, Sept. 18, 2007 Other Notable Thoughts

Need a coherent institutionalized effort Need a regular training of the entire Center staff (many unaware of PT and its benefits) Need a core group where people and projects can go for help A Way Forward

Robert J. Kuper, Executive for Reliability and Quality, Systems Engineering Conference, 23 October 2002 Utilizes Physics based behavioral model Considers inherent uncertainties, modeling uncertainties, lack of data, human error, measurement error Compensates for unknowns using statistical methods Utilizes past performance data to develop behavioral model Quantifies safety measures Qualifies prediction accuracy PT Selling Points