Presentation on theme: "Los Alamos National Laboratory Building System Models through System Ethnography First Annual Conference on Quantitative Methods & Statistical Applications."— Presentation transcript:
Los Alamos National Laboratory Building System Models through System Ethnography First Annual Conference on Quantitative Methods & Statistical Applications in Defense Andrew Koehler, PhD Alyson Wilson, PhD Christine Anderson-Cook, PhD Statistical Sciences, D-1 Los Alamos National Laboratory 2/3/2006 LA-UR-06-0951
Los Alamos National Laboratory 2 Munitions Stockpile Reliability Assessment - Summary Goal/Objective: The goal of this project is to develop a dependable and cost-effective suite of statistical methodologies and tools to assess the reliability of weapon stockpiles. Approach/Tasks Methodological development information integration uncertainty quantification with heterogeneous data Applications collaboration Tool development software for rapid development of systems and statistical models
Los Alamos National Laboratory 3 Collaborators and Customers DoD MCPD Fallbrook (TOW) NSWC Corona (RAM, ESSM) NSWC Yorktown (AMRAAM) AMCOM/RDEC (Stinger) DOE LANL Enhanced Surveillance Campaign LANL Core Surveillance
Los Alamos National Laboratory 4 The fundamental question is how to assess stockpiles as they change over time. Stockpiles change over time due to materials degradation, life-extension programs, maintenance, use, and other factors. Assessment requires the development of system models that capture parts, functions, dynamics, and interactions the integration of multiple data sources, including historical data, surveillance testing, accelerated life testing, computer model output, and materials characterization.
Los Alamos National Laboratory 5 Reliability as Currently Practiced Guidance Section Control Section Active Optical Target Detector Ordnance Section Propulsion Section Canister R AUR = R G *R C *R A * R O * R P *R Cst =0.95
Los Alamos National Laboratory 6 The (growing) Challenge Suppose that we are trying to assess a stockpile that has Multiple variants, Multiple data sources, Distributed expertise, Limits on functional testing and that we want A numerical estimate of current reliability and performance based on individual and group characteristics, A prediction of how reliability and performance change over time, Uncertainties on the estimates and predictions, perhaps as part of a capability based surveillance plan design A system description that captures stockpile environments and use dynamics.
Los Alamos National Laboratory 7 Technical Challenges Facing a multilevel data modeling and inference challenge in order to incorporate system-component surveillance data sources Keeping track of multi-level data and dependencies Existing optimal experimental design methods cannot be employed to compare the relative value of multiple, multi- level experiment types
Los Alamos National Laboratory 8 To combine the data from these different data sources, we need an approach that allows flexibility: There is a considerably variability in how much data is observed for different pieces of the system Not all components will have quality assurance data The specification limits are thought to be approximations of when the part will fail, but do not necessarily match exactly with the flight data Observed flight failure modes will not necessarily specify the failure of every component There is frequently ambiguity about which component failed during flight testing Integrating Components of Model into Unified Analysis
Los Alamos National Laboratory 9 System Ethnography a) Capturing hypotheses from all system stakeholders about what components exist in the system, and how those components relate to one another; b) Encoding component behaviors as set of rules which can tested against observed system behaviours; b) Incorporating dynamic system behaviors across all operational modes of the system; c) Linking component state information to quantitative and qualitative data sources; d) Performing checks to determine whether component reliability hypotheses are consistent and result in calculable reliability models; e) And inferring all possible combinations of component states that can result in observed system behaviors.
Los Alamos National Laboratory 10 System Ethnography and Stitching together a System Behavioral Data Model System component logic --missile descriptions and documentation --expert knowledge --existing FMECA --life-cycle/maintenance records
Los Alamos National Laboratory 11 System Ethnography and Stitching together a System Behavioral Data Model (II) Missile time- line information
Los Alamos National Laboratory 12 Fault/diagnostic/telemetry information ActivityFailure Mode Related HardwarePossible Root Cause RAM Designation Missile Not Detected ShipError in Ship Controls Ship, LauncherError in Ship to Launcher Interface LauncherError in Launcher UmbilicalError in Launcher to GMRP Interface TELEMETRY_PARAMETER Fuse AOTD (Active Optical Target Detector) Battery ELX (Electronics) Battery ELM (Electro Mechanical) Battery ELX (Electronics) Battery
Los Alamos National Laboratory 13 Using surveillance information from multiple variants can reduce uncertainty and improve prediction. We are developing the Graphical Ontology Modeling and Inference Tool (GROMIT) for system representation and qualitative inference. The gray boxes are parts or functions that appear in other variants of the system. ActRDLBPS (Front) BPS (Rear) IRUElectronics Unit Block 1ISame II Block 2ISame III ………………… Block nIISame IIVII
Los Alamos National Laboratory 14 Combine all available information to understand uncertainties in system reliability and performance. Data is often available from many different experiments: flight tests, component tests, accelerated life tests. GROMIT allows us to understand what the data tells us about the system. We also develop statistical methods to formally combine the information into a unified system reliability estimate.
Los Alamos National Laboratory 15 GROMIT allows us to combine information from different experts into an integrated system view. Different subject matter experts understand different parts of the system. GROMIT highlights potential differences in system assumptions and understanding from various experts, to create a more accurate system representation. Effective assessment requires an integrated system view.
Los Alamos National Laboratory 16 GROMIT captures the in-use dynamics and failure modes of a system. Different failure modes affect the system under various use environments giving more precise information about specific component reliabilities.
Los Alamos National Laboratory 17 GROMIT facilitates qualitative exploration of systems. Any system function or part can be set to any state and the results are propagated throughout the system to produce cut-sets. For example, if a particular failure mode is observed, we can produce a list of all combinations of parts state which might have caused this. GROMIT is not binary, but handles multiple states.
Los Alamos National Laboratory 18 System Reliability Estimate Stage 1 C1C2C3 Stage 8 C28C29C30 System What parts are in the system, how specification data links to the parts and… …the reliability information content of a particular system level outcome upon component level performance.
Los Alamos National Laboratory 19 Individual Missile Component Information is then Rolled up to Provide System Reliability System Reliability at any age is the product of all of the component reliabilities in a serial system P(system success) = function of component reliabilities
Los Alamos National Laboratory 20 Future Directions Response Space Knowledge Modeling Improved fault isolation Better characterization of continuous, non-DAG types of dependencies Stitching together analog FMECA (particularly for very large architectures)