Systems Realization Laboratory The Role and Limitations of Modeling and Simulation in Systems Design Jason Aughenbaugh & Chris Paredis The Systems Realization Laboratory The George W. Woodruff School of Mechanical Engineering The Georgia Institute of Technology November 19, 2004, Anaheim, CA IMECE

Systems Realization Laboratory Uncertainty: The Challenge of Design, Modeling, and Simulation Evaluate Alternatives Generate Alternatives Select Alternative KnowledgeInformation Generic Decision Process Predictions of Consequences of Decisions Are Always Uncertain Analyze the results Model the alternatives

Systems Realization Laboratory Uncertainty: The Challenge of Design, Modeling, and Simulation Evaluate Alternatives Generate Alternatives Select Alternative KnowledgeInformation Generic Decision Process Designers currently lack appropriate methods for representing and computing with the various types of uncertainty faced in design especially lack of knowledge

Systems Realization Laboratory Motivation: Complexity Increasing Increasingly complex Increasingly multidisciplinary Need more knowledge Need more collaboration

Systems Realization Laboratory Systems Engineering: A Decomposition Approach Forsberg, K., and Mooz, H., 1992, "The Relationship of Systems Engineering to the Project Cycle," Engineering Management Journal, 4(3), pp Forsberg, K., Mooz, H., and Cotterman, H., 2000, Visualizing Project Management: A Model for Business and Technical The Vee Model

Systems Realization Laboratory System Decomposition: Relating Requirements and Attributes RequirementsAttributesRequirementsAttributes subsystems system

Systems Realization Laboratory Relating Requirements and Attributes Engineers Decide on Engineers Design and Build RequirementsAttributesRequirementsAttributes subsystems system Have resultant Must match customer requirements

Systems Realization Laboratory Making “good” decisions Engineering Decisions Engineers Build Requirements Attributes subsystems system Have resultant Must match customer requirements ???

Systems Realization Laboratory The Role of Modeling and Simulation Engineers Decide on Engineers Build Requirements Attributes subsystems system Have resultant Must match Customer requirements Modeling and Simulation: estimates

Systems Realization Laboratory Decomposition is Hierarchical A

Systems Realization Laboratory Decomposition is Hierarchical A How do subsystem decisions affect the system attributes?

Systems Realization Laboratory Aggregation of Subsystem Attributes Depending on system composition e.g. mass Depending on system structure e.g. cost Depending on system operation e.g. reliability Resulting from complex emergent behavior e.g. queue wait times Increasingly complex Increasing value of simulation

Systems Realization Laboratory Specific Uses of Modeling and Simulation RequirementsAttributes subsystems system Models improve communication Simulations reveal emergent behaviors Models clarify requirements I didn’t think it would do that! I wanted it to behave more like… Now I understand! Models help explore robustness

Systems Realization Laboratory Limitations of Modeling and Simulation RequirementsAttributes subsystems system Representation and propagation of uncertainty Limitations of knowledge: uncertainty Integration of multiple models This is what I know: Expressing model validity So how accurate are these numbers? Is he even using the right model?

Systems Realization Laboratory How do we deal with uncertainty?  We need formalisms for Representing uncertainty accurately Computing with such formalisms Making decisions based on these formalisms  We need to accurately express what is known Capture as much of what is known as necessary Not imply information that we don’t have  Reflect different types of uncertainty

Systems Realization Laboratory Different Types of Uncertainty  Aleatory uncertainty Inherently random – irreducible Best represented as probability distribution Examples: Manufacturing variability  Epistemic uncertainty Due to a lack of knowledge Not accurately represented as probability distributions Examples: Error due to model approximation Future design decisions

Systems Realization Laboratory Possible Handling Mixed Aleatory and Epistemic Uncertainty: Probability Bounds Analysis  A p-box expresses the range of all possible CDFs that are still deemed possible based on existing knowledge. Example: An enveloping of all possible CDFs for normal distributions with variance of 1 and means in the interval [0,1] It represents aleatory uncertainty (variability) via the normal distributions It represents epistemic uncertainty (incertitude) via the interval on the parameters A “P-box” N(0,1) N(1,1)

Systems Realization Laboratory P-boxes: two dimensions of uncertainty Variable Deterministic Epistemic Precise

Systems Realization Laboratory Summary: we need more appropriate representations of uncertainty Evaluate Alternatives Generate Alternatives Select Alternative KnowledgeInformation Generic Decision Process Predictions of Consequences of Decisions Are Always Uncertain Analyze the results Model the alternatives

Systems Realization Laboratory Summary: we need more appropriate representations of uncertainty Evaluate Alternatives Generate Alternatives Select Alternative KnowledgeInformation Generic Decision Process Predictions of Consequences of Decisions Are Always Uncertain Analyze the results Model the alternatives Better Representations Better Selection Better Design

Systems Realization Laboratory Acknowledgements  Thank you for attending!  This material is based upon work supported under a National Science Foundation Graduate Research Fellowship. Any opinions, findings, conclusions or recommendations expressed in this presentation are those of the authors and do not necessarily reflect the views of the National Science Foundation.  Additional support is provided by the G.W. Woodruff School of Mechanical Engineering at Georgia Tech.

Systems Realization Laboratory Questions?