1. Overview of Reliability Analysis and Design Capabilities in DAKOTA
Michael S. Eldred
Sandia National Laboratories
Barron J. Bichon
Vanderbilt University
Brian M. Adams
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000.

2. Introduction
Uncertainties/variabilities must be properly modeled in order to quantify risk and design systems that are robust and reliable
Uncertainty comes in different flavors:
Aleatory/irreducible: inherent uncertainty/variability with sufficient data  probabilistic models
Epistemic/reducible: uncertainty from lack of knowledge  nonprobabilistic models
We use a UQ-based approach to optimization under uncertainty (OUU)
safety factors, multiple operating conditions, or local sensitivities are insufficient
Tailor OUU methods to strengths of different UQ approaches
OUU methods encompass both:
design for robustness (moment statistics: mean, variance)
design for reliability (tail statistics: probability of failure)
Harder problem

3. Introduction (cont.)
Focus is simulation-based engineering applications
mostly PDE-based, often transient
response mappings are nonlinear and implicit
distinct from chance-constrained stochastic programming (often linear/explicit)
Augment with general response statistics su (e.g., m, s, z/b/p) using a linear mapping:
Minimize f(d) + Wsu(d)
Subject to gl  g(d)  gu
h(d) = ht
al  Aisu(d)  au
Aesu(d) = at
dl  d  du
[Nonlinear mappings (s2, s/m, etc.) via AMPL]
Standard NLP:
Minimize f(d)
Subject to gl  g(d)  gu
h(d) = ht
dl  d  du

4. Outline G(u) Introduction Algorithms
DAKOTA software
Uncertainty quantification
Overview
Reliability methods
Sample benchmark results
Optimization under uncertainty
RBDO methods
Shape Optimization of Compliant MEMS
Bi-stable switch
RF switch
Concluding Remarks

5. DAKOTA Overview Goal: answer fundamental engineering questions
Optimization
Uncertainty Quant.
Parameter Est.
Sensitivity Analysis
Black Box: Sandia simulation codes
Commercial simulation codes
Semi-instrusive:
SIERRA (multiphysics),
SALINAS (structural dynamics),
Xyce (circuits), Sage (CFD),
MATLAB, Mathematica, ModelCenter, FIPER
Model
Parameters
Design
Metrics
Nominal
Optimized
Goal: answer fundamental engineering questions
What is the best design?
How safe is it?
How much confidence in my answer?
Technical themes
Large-scale optimization, UQ, and V&V
Algorithms for complex engineering applications: SBO, OUU
Balance algorithm research with production demands
Impact
Internal: Large-scale ASC applics., multiphysics framework deployment
External: Tri-lab, WFO, GPL open source (~3000 download registrations)

6. DAKOTA Framework Iterator LeastSq DoE UQ ParamStudy Optimizer
Model:
Parameters
Parameters
Interface
Responses
LeastSq
DoE
Design continuous discrete
Application system fork direct grid
Functions
DDACE
CCD/BB
NLSSOL GN
objectives
QMC/CVT
NL2SOL
constraints
Uncertain normal/logn uniform/logu triangular beta/gamma EV I, II, III histogram interval
least sq. terms UQ
ParamStudy
generic
Optimizer
Approximation global polynomial 1/2/3, NN, kriging, MARS, RBF multipoint – TANA3 local – Taylor series hierarchical
Gradients
LHS/MC
DSTE
Vector
List
numerical analytic
Reliability
SFEM
Center
MultiD
Hessians
numerical analytic quasi
State continuous discrete
DOT
CONMIN
NPSOL
NLPQL
OPT++
COLINY
JEGA
Strategy: control of multiple iterators and models
Coordination:
Strategy
Iterator
Nested
Layered
Cascaded
Concurrent
Adaptive/Interactive
Model
Optimization
Uncertainty
Iterator
Parallelism:
Hybrid
OptUnderUnc
Model
Asynchronous local
Message passing
Hybrid
4 nested levels with
Master-slave/dynamic
Peer/static
SurrBased
UncOfOptima
Iterator
Pareto/MStart
2ndOrderProb
Branch&Bound/PICO
Model

7. Uncertainty QuantificationUQ
LHS/MC
DSTE
Reliability
SFEM
Active UQ development (new, developing, planned).
Sampling: LHS/MC, QMC/CVT, Bootstrap/Importance/Jackknife. Gunzburger collaboration.
Reliability: MVFOSM, x/u AMV, x/u AMV+, FORM (RIA/PMA mappings), MVSOSM, x/u AMV2, x/u AMV2+, TANA, SORM (RIA/PMA) Renaud/Mahadevan collaborations.
SFE: Polynomial chaos expansions (quadrature/cubiture extensions). Ghanem (Walters) collaborations.
Metrics: Importance factors, partial correlations, main effects, and variance-based decomposition.
Epistemic: 2nd-order probability, Dempster-Schafer, Bayesian.
Quick transition: DAKOTA/UQ continues to be a highly active area and a big programmatic driver.
Have already touched on new 2nd-order reliability methods, will highlight new epistemic methods d
Uncertainty applications: penetration, joint mechanics, abnormal environments, shock physics, …

8. Epistemic UQ Second-order probability
Two levels: distributions/intervals on distribution parameters
Outer level can be epistemic (e.g., interval)
Inner level can be aleatory (probability distrs)
Strong regulatory history (NRC, WIPP).
Dempster-Schafer theory of evidence
Basic probability assignment (interval-based)
Solve opt. problems (currently sampling-based) to compute belief/plausibility for output intervals
[Also, could do basic black-box response interval estimation with min/max global optimization]
2005
2006

9. UQ with Reliability Methods
Mean Value Method
Surprisingly popular with analysts
Reliability Index Approach (RIA)
Performance Measure Approach (PMA)
Find min dist to G level curve
Used for fwd map z  p/b
Find min G at b radius
Better for inv map p/b  z
G(u)
MPP search methods
Variations: MPP search alg, Approximations, Integrations, Warm starting …

10. Reliability Algorithm Variations: First-Order Methods
Limit state linearizations
AMV:
u-space AMV:
AMV+:
u-space AMV+:
FORM: no linearization
Integrations
1st-order:
MPP search algorithm
[HL-RF], Sequential Quadratic Prog. (SQP), Nonlinear Interior Point (NIP)
Warm starting
When: AMV+ iteration increment, z/p/b level increment, or design variable change
What: linearization point & assoc. responses (AMV+) and MPP search initial guess
RIA Projection:
PMA Projection:
MPP initial guess benefits from projection since KKT conditions w.r.t. u still satisfied for new level at previous optimum
Eldred, M.S., Agarwal, H., Perez, V.M., Wojtkiewicz, S.F., Jr., and Renaud, J.E., Investigation of Reliability Method Formulations in DAKOTA/UQ, (to appear) Structure and Infrastructure Engineering, Taylor & Francis.

11. Reliability Algorithm Variations: Second-Order Methods
2nd-order local limit state approximations
e.g., x-space AMV2+:
Hessians may be full/FD/Quasi
Quasi-Newton Hessians may be BFGS or SR1
G(u)
Failure region
Multipoint limit state approximations
e.g., TPEA, TANA:
Synergistic features:
Hessian data needed for SORM integration can enable more rapid MPP convergence
[QN] Hessian data accumulated during MPP search can enable more accurate probability estimates
2nd-order integrations
Also, AIS, …
curvature correction
Eldred, M.S. and Bichon, B.J., Second-Order Reliability Formulations in DAKOTA/UQ, (in review) Structure and Infrastructure Engineering, special issue on uncertainty in aerospace systems, Taylor & Francis.

12. Reliability Algorithm Variations: Sample of Results to Date
Analytic benchmark test problems: lognormal ratio, short column, cantilever
43 z levels
43 p levels
Limit state surrogate approaches are significantly more effective than general purpose optimizers (which may use internal linear/quadratic approxs.)
Best (practical) performer to date:
AMV2+ (with SR1 Hessian updates)
More efficient (RIA, PMA) and more robust (PMA with 2nd-order p levels)
Note: 2nd-order PMA with prescribed p level is harder problem  requires b(p) update/inversion

13. Outline G(u) Introduction Algorithms
DAKOTA software
Uncertainty quantification
Overview
Reliability methods
Sample benchmark results
Optimization under uncertainty
RBDO methods
Shape Optimization of Compliant MEMS
Bi-stable switch
RF switch
Concluding Remarks

14. Optimization Under Uncertainty
OUU techniques categorized based on UQ approach:
Sampling-based (noise-tolerant opt.; design for robustness)
TR-SBOUU: trust region surrogate-based
Nongradient-based (Trosset)
Robust design of experiments (Taguchi)
Reliability-based (exploit structure; design for reliability)
Bi-level RBDO (nested)
Sequential RBDO (iterative)
Unilevel RBDO (all at once)
Stochastic finite element-based (multiphysics)
Exploit PCE coeffs & random process structure
Leverage SBO with global surrogates
Epistemic uncertainty
Evidence theory-based (Agarwal)
Bayesian inference: model calibration under uncertainty
2nd-order probability: 3-level OUU approaches
Intrusive OUU (all at once approaches)
SFE + SAND: intrusive PCE variant amenable to SAND
Unilevel RBDO + SAND
Opt UQ
{d} {S u }
Data Fit
Sim
{u} {R
Data Fit/Hier
2003
Augment NLP with response statistics su (m, s, p/b/z) using a linear mapping:
Minimize f(d) + Wsu(d)
Subject to gl  g(d)  gu
h(d) = ht
al  Aisu(d)  au
Aesu(d) = at
dl  d  du
2006

15. RBDO Algorithms Bi-level RBDO Fully analytic Bi-level RBDO
Constrain RIA z  p/b result
Constrain PMA p/b  z result
RIA RBDO
PMA RBDO
Fully analytic Bi-level RBDO
Analytic reliability sensitivities avoid numerical differencing at design level
(1st order)
If d = distr param, then expand
1st-order (also 2nd-order, …)
Sequential/Surrogate-based RBDO:
Break nesting: iterate between opt & UQ until target is met. Trust-region surrogate-based approach is non-heuristic.
KKT
of MPP
Unilevel RBDO:
All at once: apply KKT conditions of MPP search as equality constraints
Opt. increases in scale (d,u)
Requires 2nd-order info for derivatives of 1st-order KKT

16. RBDO Algorithm Variations: Sample of Results to Date
Analytic benchmark test problems: short column, cantilever, steel column
Short Column min bh s.t. b > 2.5
P = N(500, 100) rP,M = 0.5
M = N(2000, 400) bnom = 5
Y = LogN(5, 0.5) hnom = 15
Kuschel & Rackwitz, 1997

17. OUU Progress To Date 2003: Surrogate-based OUU with sampling methods
2004: Bi-level RBDO with numerical reliability gradients
2005: Fully analytic bi-level RBDO Sequential/surrogate-based RBDO (1st-order, 2nd-order)
OUU/SBOUU
RBDO BL S1
FA-BL
Two orders of magnitude
improvement (so far)
over “brute force” OUU
With tuning of initial TR size, 3 RBDO benchmarks solved in ~40 fn evals per limit state:
35 for 1 limit state in short column
75 for 2 limit states in cantilever
45 for 1 limit state in steel column S2

18. Outline G(u) Introduction Algorithms
DAKOTA software
Uncertainty quantification
Overview
Reliability methods
Sample benchmark results
Optimization under uncertainty
RBDO methods
Shape Optimization of Compliant MEMS
Bi-stable switch
RF switch
Concluding Remarks

19. Engineering Application Deployment: Shape Optimization of Compliant MEMS
MEMS designs are subject to substantial variabilities and lack historical knowledge base
Sources of uncertainty
Material properties, manufactured geometries, residual stresses
Data can be obtained  aleatoric uncertainty, probabilistic approaches
Resulting part yields can be low or have poor cycle durability
Goals:
Achieve prescribed reliability
Minimize sensitivity to uncertainties (robustness)
Nonlinear FE simulations
~20 min. desktop simulation expense (SIERRA codes: Adagio, Aria, Andante)
Remeshing with FASTQ/CUBIT or smooth mesh movement with DDRIV
(semi-analytic) p/b/z gradients appear to be reliable
Bi-stable MEMS Switch
RF MEMS Switch

20. Bi-Stable Switch: Problem Formulation
13 design vars: Wi, Li, qi
2 random vars: m)
 reliable + robust

21. Bi-Stable Switch: Results
MVFOSM-based RBDO
AMV+/FORM-based RBDO
Reliability: target achieved for AMV+/FORM; target approximated for MV
Robustness: variability in Fmin reduced from 5.7 to 4.6 mN per input s [mFmin/b]
Continuing: quantity of interest error estimates  error-corrected UQ/RBDO

22. Bi-Stable Switch: Observed Challenges
MVFOSM-based RBDO d*
Problematic d
Smooth, linear limit state (in range of interest)
First-order integration OK
Nonlinear limit state
Higher-order integration needed
Nonsmooth due to sim failures in left tail of edge bias (flimsy structure)
In general, need support of nonlinear/nonsmooth/multimodal limit states and careful attention to problem formulation (e.g., supporting simulation requirements)

23. Conclusions & Future Directions
General UQ/OUU Aspects:
Nonlinear, large-scale, expensive simulations w/ implicit, noisy response metrics
Aleatory and epistemic uncertainties
Design for robustness and reliability
Reliability Analysis:
Explored limit state approxs, integrations, MPP search algs, Hessian approxs, warm starting
New algorithms for 2nd-order PMA, AMV2/AMV2+, and QN integrations
Recommendation for AMV2+ (with SR1 limit state Hessian updates)
Future work: noisy limit states, multiple MPPs  global surrogates, TR approaches, global opt.
RBDO Algorithms:
Explored bi-level and sequential methods exploiting probabilistic sensitivities
New sequential RBDO approaches employing TRs and 2nd-order local approxs.
Recommendation for 2nd-order sequential RBDO (with SR1 metric Hessian updates)
Future work:
enhancements to input distr. parameterization and output distr. characterization
model calibration/inversion under uncertainty (matching output CDFs)
explore TR linkages between MPP and design levels

24. Extra Slides

25. Optimization with Surrogate Models
Purpose:
Reduce the number of expensive, high-fidelity simulations by using a succession of approximate (surrogate) models
Approximations generally have a limited range of validity
Trust regions adaptively manage this range based on efficacy during opt
With trust region globalization and local 1st-order consistency, SBO algorithms are provably-convergent
Surrogate models of interest:
Data fits
Multifidelity (special case: multigrid optimization)
Reduced-order models
Future connections to multi-scale for managing approximated scales

26. Trust-Region Surrogate-Based Optimization
Data Fit
Data fit surrogates:
Global: polynomial regress., splines, neural net, kriging, radial basis fn
Local: 1st/2nd-order Taylor
Multipoint: TANA, …
Data fits in SBO
Smoothing: extract global trend
DACE: number of des. vars. limited
Local consistency must be balanced with global accuracy
Multifidelity surrogates:
Coarser discretizations, looser conv. tols., reduced element order
Omitted physics: e.g., Euler CFD, panel methods
Multifidelity SBO
HF evals scale better w/ des. vars.
Requires smooth LF model
Design vector maps may be reqd.
Correction quality is crucial
 Multifidelity
ROM surrogates:
Spectral decomposition (str. dynamics)
POD/PCA w/ SVD (CFD, image analysis)
KL/PCE (random fields, stoch. proc.)
RBGen/Anasazi
ROMs in SBO
Key issue: capture parameter changes
Extended ROM, Spanning ROM
Shares features of data fit and multifidelity cases
New area
 ROM
ROM area has some attractive advantages, in that it is more physics-based than a regression-based data fit, and does not require multiple computational models or meshes which are not always available/practical. But a key issue that must be solved is capturing of parameter changes in the ROM.

27. SBOUU Formulations For surrogate-based OUU, the surrogate can appear
at the optimization level (fit S(d))
at the UQ level (fit R(d, u))
at both levels (fit S(d) and R(d, u))
Surrogate can be
local/global/multipoint data fit (either level)
model hierarchy approximation (UQ level only)
Opt UQ
{d} {S u }
Data Fit
Sim
{u} {R
Data Fit/Hier
SBOUU with two surrogate levels:
Simple Nested OUU:
TO DO: add ROM case to this

28. Optimization under Uncertainty with Surrogates
DakotaModel
Single
Layered
Nested
Data Fit
Hierarchical
Nested model: internal iterators/models execute a complete iterative study as part of every evaluation.
Surrogate model: internal iterators/models used for periodic update and verification of data fit (global/local/multipoint) or hierarchical (variable fidelity) surrogates.
Nested/Surrogate models can recurse
Formulation 1: Nested
Opt UQ
{d} {S u }
Data Fit
Sim
{u} {R
Data Fit/Hier
Opt UQ
Sim
{d} {S u }
{u} {R
Data Fit
Opt UQ
Sim
{d} {S u }
{u} {R
Data Fit/Hier
Formulation 2: Surrogate
containing Nested
Formulation 3: Nested
containing Surrogate
Formulation 4: Surrogate
containing Nested containing Surrogate
Formulations 2 & 4 amenable to trust-region approaches
Goals: maintain quality of results, provable convergence (for a selected confidence level)

29. TR-SBOUU Results Minimize f + pfail_r1 + pfail_r3
Direct nested OUU is expensive and requires seed reuse
SBOUU expense much lower (up to 100x), but unreliable.
TR-SBOUU maintains quality of results and reduces expense ~10x
Ex. 1: formulation 4 with TR 5-7x less expensive than direct nesting
Ex. 2: formulation 4 with TR 8-12x less expensive than direct nesting
ICF Ex.: formulations 2/4 with TR locate vicinity of a min in a single cycle
Additional benefits:
Navigation of nonsmooth engineering problems
Less sensitive to seed reuse: variable patterns OK and often helpful, possibility of exploitation reduced
Less sensitive to starting point: data fit SBO provides some global ident.
Minimize f + pfail_r1 + pfail_r3
Subject to gi  0, for i = 1,2,3
mr2 + 3sr2  1.6e5
Conference papers at AIAA MA&O, SIAM CS&E, USNCCM:
Eldred, M.S., Giunta, A.A., Wojtkiewicz, S.F., Jr., and Trucano, T.G., "Formulations for Surrogate-Based Optimization Under Uncertainty."

30. Trust Region Surrogate-Based Optimization under Uncertainty (TR-SBOUU)
From SBO to SBOUU:
SBO is provably convergent with TR globalization
(at least) 1st order consistency with correction
verification of approx. steps
Extensions to SBOUU
1st order consistency, assuming a worthwhile stoch. gradient
verification of stats. in relative sense. Three levels of verification rigor:
Least: nominal statistics.
Most: ordinal opt. (Chen/Romero)  nonoverlap confidence bounds on every step (“provable” convergence for a selected confidence level).
Affordable compromise: stochastic approximation (Igusa)  probability of erroneous TR steps is decreased in proportion to iteration count.
Sequence of trust regions

31. Intrusive OUU: DAKOTA/MOOCHO w/ SIERRA/NEVADA
Next-generation multi-physics simulation architectures:
SIERRA: mechanics framework (“S. DAKOTA”)
NEVADA: physics framework (“N. DAKOTA”)
Architecture extensions underway for:
Opt.: SAND optimization (MOOCHO)
UQ: (intrusive) stochastic finite elements
Other: Stability analysis (LOCA), Nonlinear equations (NOX), Fully-coupled MDA
Impact:
Performance enhancements for existing nested methods
Model I/O in core in parallel
SPMD execution on compute nodes of ASCI MPPs
Next-generation, tightly-coupled opt. & UQ
Direct/adjoint sensitivities & AD
Sierra
Common Code Ar c
hitecture
shock
physics
transient dynamics
Incomp
Fluids
structural
dynamics
comp fluid
Fire
DSMC
Opt./UQ
Solid
Mech
Thermal
radiation
Dakota
Optimization
Uncertainty Quantification
Parameter Estimation
Sensitivity Analysis
MOOCHO
Veltisto
O3D
SFE
Intrusive OUU:
SFE + SAND
Unilevel RBDO + SAND

32. RBDO Results Cantilever min wt s.t. bD, bS > 3
Limit state eqns (unnormalized):
Wu et al., 2001
2 design vars: w, t
4 uncorr. normal uncertain vars: E, R, X, Y

33. Concluding Remarks
DAKOTA/UQ provides a flexible object-oriented framework for investigating algorithmic variations in reliability analysis and RBDO
Novel aspects:
Full CDF/CCDF
Limit state linearizations in PMA
Warm starting by projection (avoids premature convergence)
SQP vs. NIP
UQ Performance:
Relative to FORM:
MV 2 orders of magnitude reduction but only accurate near means
AMV 1 order of magnitude reduction but MPP not converged
AMV+ ~3x reduction with full accuracy
Warm starts: significant win, as expected
NIP vs. SQP: mixed results, but NIP promising (can avoid u-space excursions)
x-space vs. u-space: mixed results, application dependent
RBDO Performance:
MV/AMV RBDO is cheap and gets in the solution vicinity
RIA zb RBDO: preferred to RIA zp RBDO, slight advantage over PMA RBDO
AMV+ RBDO 3.5x more efficient than FORM RBDO
Great for rough stats