1. Joseph P. Castro Jr. *, Genetha Gray , Anthony Giunta , Patricia Hough , and Paul Demmie  Sandia National Laboratories: * Computational Sciences,  Computational Physics/Simulation Frameworks,  Validation & Uncertainty Quantification Processes,  Computational Sciences & Math 2005 SIAM Conference on Computational Science and Engineering February 13, 2005 Orlando, FL *Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL85000. A New Scheme for Multifidelity Optimization Incorporating Pattern Search and Space Mapping

2. Overview Motivation Description of Multifidelity Optimization Scheme –Asynchronous Parallel Pattern Search (APPS) description –Space Mapping description –APPS/Space Mapping optimization scheme Space Mapping Sensitivity Study Penetrator Example Future Work Summary

3. Motivation for a Multifidelity Optimization Approach In many optimization applications, the objective function f(x) is expensive to calculate and derivatives may be inaccurate if they exist at all. Many of these applications have a high fidelity (or true) model and a low fidelity (or surrogate) model which simplifies the high fidelity model in some way. An MFO approach optimizes an inexpensive, low fidelity model while making periodic corrections using the expensive, high fidelity model. Add low fidelity approximation of graphic

4. Low Fidelity 30,000 DOF High Fidelity 800,000 DOF Finite Element Models of the Same Component Multifidelity Surrogate Models The low-fidelity surrogate model retains many of the important features of the high-fidelity “truth” model, but is simplified in some way. –decreased physical resolution –decreased FE mesh resolution –simplified physics An MFO approach optimizes an inexpensive, low fidelity model while making periodic corrections using the expensive, high fidelity model. Works well when low-fidelity trends match high-fidelity trends.

5. parent node child node pruned node dislocated node APPS Allows Us To Integrate a Low-Fidelity Response For Multifidelity Optimization Pattern search is a non-gradient optimization search with pre-determined patterns. Asynchronous Parallel Pattern Search (APPS)*, takes advantage of non-dependent responses with very different compute times –Ideal fit for use with multifidelity optimization APPSPACK is open source software that implements the APPS algorithm –Does not assume homogeneous processors (MPI implementation) *Developed by Patricia Hough, Tamara Kolda, Virginia Torczon

6. Space mapping* is a technique that maps the design space of a low fidelity model to the design space of high fidelity model such that both models result in approximately the same response. The parameters within x H need not match the parameters within x L Space Mapping* Provides a Conduit Between The Design Spaces of the Low and High Fidelity Models x – design variables R - response P - mapping xHxH RH(xH)RH(xH) high-fi model xLxL RL(xL)RL(xL) low-fi model *Developed by John Bandler, et. al. xHxH R L (P(x H )) mapped low-fi model P(x H ) xL=P(xH)xL=P(xH) R L (P(x H ))  R H (x H ) such that We’re using the mapping ?

7. The APPS/Space Mapping Scheme Outer Loop Inner Loop Low Fidelity Model Optimization  x H    High Fidelity Mode Optimization via APPSPACK Space Mapping Via Nonlinear Least Squares Calculation  multiple x H,f(x H ) x H trial

8. An Oracle Provides a Way of Generating Another Search Point Outside of Pattern Search If the inner loop is viewed as an oracle, the only significant algorithmic change to APPS is the addition of the oracle. Analytically, there are no restrictions on how an oracle can choose points. Oracle points are used in addition to the points defined by the search pattern The convergence APPSPACK is not adversely affected. A convergence proof for the APPS algorithm exists so no new convergence proof for our MFO scheme is needed. Future work may include investigating any improvement to the convergence of APPS.

9. A Simple Polynomial was Used to Study Space Mapping Sensitivities Ideally     *,     *, etc... (f H = f mapped ) Studied the space mapping sensitivities to various inputs –# high fidelity responses used for the mapping –scaling of the mapping parameters (size of offset between the low and high fidelity models) –starting point Compare the optimum found and the number of high fidelity runs required to reach the optimum High Fidelity Model: Low Fidelity Model: Mapped Space (  *,  *,  * calculated via Least Squares):

10. # responses (x 0,x 1 ) (-0.5,0.83) Objective Value 0.0 # Hi-Fi Calculations X Speed Up 2 (-0.50, 0.83) 3.12e-10133.31 4 (-0.50, 0.83) 3.12e-10172.52 6 (-0.49, 0.81) 4.27e-4272.52 8 (-0.48, 0.81) 1.03e-3271.59 Apps only (-0.40, 0.80) 1.09e-243-  ~  ;  =1 Starting Point = (-2.0,-2.0) The APPS/Space Mapping Scheme Improved Optimization Performance and Value

11. Plot of Best Points Found With APPS/Space Mapping Scheme Polynomial Model with  ~O(1);  =1, starting point = (-2,-2) 13 17 27 43 In all cases the inner loop call finds a best point with the first call In all cases the inner loop call finds a best point with the first call All inner loop calls beyond this do not find a best point (APPS dominates at this point) All inner loop calls beyond this do not find a best point (APPS dominates at this point)

12. View of High Fidelity Design Space View of Unmapped Low Fidelity Design Space Comparison of Design Space of High and Low Fidelity Polynomial Models with ,  ~O(1);  =1

13. Plot of Best Points Found With APPS/Space Mapping Scheme Polynomial Model with  ~O(1);  =1 starting point = (-2,-2) Plot of Best Points Found With APPS/Space Mapping Scheme Polynomial Model with  ~O(1);  =1, starting point = (-2,-2) 47 51 34 53 Though there is an improvement with the inner loop, the performance is not as great as with the previous case Though there is an improvement with the inner loop, the performance is not as great as with the previous case The APPS only case had the best optimal value as wellThe APPS only case had the best optimal value as well

14. 1 Best Case: # response points = 8 2 calls to inner loop Approximate Inner Loop Call Locations within Hi-Fi Model (-0.76,2.0) (-0.8,-1.2) 1 2 The numbered white boxes show approximately where the inner loop was called The point in red brackets is where APPSPACK is before the inner loop call The point in green was found by the inner loop 2 (-0.56,1.6) (-0.61,1.25)

15. 1 Worst Case: # response points = 6 (5 calls to inner loop) 2 3 4 5 Hi-Fi Model (-0.87,.006) (-0.8,-1.2) (-0.47,0.39) (-0.47,1.19) (-0.67,1.60) (-0.67,1.71) (-0.35,-.013) (-0.53,0.29) (-0.64,1.36) (-0.63,1.32) 1 2 3 4,5

16. A computational cost ratio of 1:240 The low-fidelity model gives the same general response trends as the high- fidelity model These factors makes these models prime candidates for multifidelity optimization V  pp cc NN Penetrator Case: 3-D Model of Steel Earth Penetrator Striking a Concrete Target Steel Penetrator composed of multiple materials entering a concrete target High Fidelity Model = elastic EP body –~40 minute calculation time Low Fidelity Model = rigid EP body –~10 second calculation time Target -y

17. Acceleration Comparison with Varying Mesh Rigid body response follows the trend of elastic body response Fine Mesh Coarse Mesh # Elements 78481152 Elastic Calc. Time (s) 2400180

18. Minimize Acceleration with Varying Density Hi-Fidelity Model = Elastic Model (Fine Mesh) Low-Fidelity Model = Rigid Model (Fine Mesh) A series of calculations were done minimizing acceleration and maximizing displacement  displacement 2-3x speed up  acceleration 1-2x speed up For this case, the APPS/Space Mapping scheme took longer to converge but a better optimum was found provides a type of global search capability to get past the local “noise”

19. Ongoing and Future Work Study spaces defined using different constraints. Implement a generic oracle in APPSPACK. Include a space mapping that does not require domain spaces to be defined by the same numbers of parameters. Apply our multifidelity optimization schemes to some real world problems:  Earth penetrator analysis  Groundwater problems including well field design & hydraulic capture  Circuit system design

20. References and Contact Information APPSPACK: Software Website APPSPACK 4.0 http://software.sandia.gov/appspack/version4.0http://software.sandia.gov/appspack/version4.0 This website includes the software itself (open-source) and instructions for downloading, installing, and using it. It also has a complete list of references to papers on the software development and convergence analysis. DAKOTA: Software Website http://endo.sandia.gov/DAKOTA ORACLE: Overview Paper Kolda, Lewis, and Torczon, "Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods," SIAM Review, 45(3):385-482, 2003. SPACE MAPPING: Bakr, Bandler, Madsen, and Sondergard, "An Introduction to Space Mapping Technique, " Optimization & Engineering, 2:369-384, 2001. Contact Info: Joseph Castro: jpcastr@sandia.gov