1. Design Optimization Utilizing Gradient/Hessian Enhanced Surrogate Model Dept. of Mechanical Engineering, University of Wyoming, USA Wataru YAMAZAKI, Markus P. RUMPFKEIL, Dimitri J. MAVRIPLIS 28 th, June, 2010, 28 th AIAA Applied Aerodynamics Conference

2. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -2- Outline *Background - Efficient CFD Gradient/Hessian calculations - Surrogate Model Enhanced by Gradient/Hessian - Uncertainty Analysis *Objectives *Surrogate Model Approaches - Kriging - Direct and Indirect Gradient-enhanced Kriging - Gradient/Hessian-enhanced Kriging Approaches *Results & Discussion - Analytical Function Fitting - Aerodynamic Data Modeling - 2D Airfoil Drag Minimization - Uncertainty Analysis at Optimal Airfoil *Conclusions

3. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -3- Background~ Efficient CFD Hessian Calculation M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268 “Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method” An efficient CFD Hessian calculation method by Adjoint method and Automatic Differentiation (AD) For steady flow i. Solutions for grid deformation / flow residual equations ii. Adjoint solutions for flow / grid deformation equations iii. N dv linear solutions each for dx/dD j and dw/dD j iv. N dv (N dv +1)/2 cheap evaluations for each Hessian component

4. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -4- Background~ Efficient CFD Hessian Calculation Grid Deformation Flow Residual Flow Adjoint Mesh Adjoint dx/dD 1 dw/dD 1 dx/dD 2 dw/dD 2 dx/dD Ndv dw/dD Ndv...... Gradient and Hessian An efficient CFD Hessian calculation method by Adjoint method and Automatic Differentiation (AD) M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268 “Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method”

5. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -5- Background~ Approximate CFD Hessian For steady flow, a special form of objective function Last approximation is accurate only nearly optimum Approximate Hessian only requires the first-order derivatives M.P. Rumpfkeil and D.J. Mavriplis, AIAA-2010-1268 “Efficient Hessian Calculations using Automatic Differentiation and the Adjoint Method”

6. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -6- Background~ Uncertainty Analysis Uncertainty due to manufacturing tolerances in-service wear-and-tear etc Analysis of mean/variance/PDF of objective function w.r.t. fluctuation of design variables Full Monte-Carlo Simulation Thousands/Millions exact function calls Accurate and easy, but computationally expensive Moment Method Taylor series expansion by grad/Hessian at the center No information about PDF Inexpensive Monte-Carlo Simulation Thousands/Millions surrogate model function calls Much lower computational cost

7. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -7- Objectives The efficient adjoint gradient/Hessian calculation methods will be effective… for more efficient global design optimization with G/H-enhanced surrogate model approach for more accurate and cheaper uncertainty analysis by inexpensive Monte-Carlo simulation with G/H-enhanced surrogate model Development of gradient/Hessian-enhanced surrogate models Application to design optimization and uncertainty analysis

8. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -8- Kriging, Gradient-enhanced Kriging Kriging model approach - originally in geological statistics Two gradient-enhanced Kriging (cokriging or GEK) Direct Cokriging Gradient information is included in the formulation (correlation between func-grad and grad-grad) Indirect Cokriging Same formulation as original Kriging Additional samples are created by using the gradient info Kriging model by both real and additional pts 2D example : Real Sample Point : Additional Sample Point

9. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -9- Gradient/Hessian-enhanced Kriging Indirect Approach 2D example : Real Sample Point : Additional Sample Point Arrangements to Use Full Hessian / Diagonal Terms Major parameters : distance between real / additional pts number of additional pts per real pt Worse matrix conditioning withsmaller distance larger number of additional pts Severe tradeoffs for these parameters

10. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -10- Gradient/Hessian-enhanced Kriging Direct Approach Consider a random process model estimating a function value by a linear combination of function/gradient/Hessian components Minimizing Mean-Squared-Error (MSE) between exact/estimated function with an unbiasedness constraint Solving by using the Lagrange multiplier approach

11. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -11- Gradient/Hessian-enhanced Kriging Direct Approach Introducing correlation function for covariance terms Correlation is estimated by distance between two pts with radial basis function Unknown parameters are determined by the following system of equations Final form of the gradient/Hessian-enhanced direct Kriging approach is

12. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -12- Gradient/Hessian-enhanced Kriging Direct Approach Correlations between F-F, F-G, G-G, F-H, G-H and H-H Up to 4 th order derivatives of correlation function Automatic Differentiation by TAPENADE No sensitive parameter Better matrix conditioning than indirect approach

13. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -13- Infill Sampling Criteria for Optimization How to find promising location on surrogate model ? Maximization of Expected Improvement (EI) value Potential of being smaller than current minimum (optimal) Consider both estimated function and uncertainty (RMSE)

14. Results & Discussion

15. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -15- 2D Rastrigin Function Fitting 80 samples by Latin Hypercube Sampling Direct Kriging approach Exact Rastrigin FunctionFunction-based KrigingGradient-enhancedGradient/Hessian-enhanced

16. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -16- 5D Rosenbrock Function Fitting F: Function-based Kriging FG: Gradient-enhanced FGHd: G/diag. Hess-enhanced FGH: G/full Hess-enhanced RMSE.vs. Number of sample points Superiority in direct Kriging approaches thanks to exact enforcement of derivative information better conditioning of correlation matrix

17. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -17- Validation on Rosenbrock Func. Minimization of 20D Rosenbrock 30 initial sample points by LHS EI-based infill sampling criteria Faster convergence in G/H-enhanced direct approach Uncertainty analysis on 2D Rosenbrock 5 sample points for surrogate model (No sample point on the center location) Superior performance in G/H-enhanced Inexpensive MC (IMC) CDFs of Full-MC and IMC Optimization History

18. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -18- Aerodynamic Data Modeling Unstructured mesh CFD Steady inviscid flow, NACA0012 20,000 triangle elements Mach Number [0.5, 1.5] Angle of Attack [deg] [0.0, 5.0] 21 x 21=441 validation data Exact Hypersurface of Lift CoefficientExact Hypersurface of Drag Coefficient

19. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -19- Aerodynamic Data Modeling Adjoint gradient is helpful to construct accurate surrogate model CFD Hessian is not helpful due to noisy design space Function-based KrigingGradient-enhancedExact ClCl CdCd

20. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -20- 2D Airfoil Shape Optimization Unstructured mesh CFD Steady inviscid flow, M=0.755 NACA0012, 16 DVs for Hicks-Henne function Objective function of inverse design form Exact / Approximate CFD Hessian available Computational time of F : 2 min, FG : 4 min, FGH approx. : 6 min, FGH exact : 36 min (4 min in parallel) Geometrical constraint for sectional area

21. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -21- 2D Airfoil Shape Optimization Start from 16 initial sample points which only have function info Gradient/Hessian evaluations only for new optimal designs Faster convergencein derivative-enhanced surrogate model Best design in gradient/exact Hessian-enhanced model

22. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -22- 2D Airfoil Shape Optimization Towards supercritical airfoils Shock reduction on upper surface NACA0012 (Baseline)Optimal by G/exact H-enhanced model

23. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -23- 2D Airfoil Shape Uncertainty Analysis Geometrical uncertainty analysis at optimal airfoil Center = optimal obtained by Grad/exact H model Comparison between 2 nd order Moment Method (MM2) using gradient/Hessian at the center Inexpensive Monte-Carlo (IMC1) using final surrogate model obtained in optimization Inexpensive Monte-Carlo (IMC2) using different G/H-enhanced model by 11 samples Full Non-Linear Monte-Carlo (NLMC) using 3,000 CFD function calls optimal (center) ±0.1 airfoil

24. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -24- 2D Airfoil Shape Uncertainty Analysis Mean of objective w.r.t. standard deviation of all design variables IMC showed good agreement with NLMC at smaller st. devi. Necessity of additional sampling criteria for total model accuracy ? Promising IMC with much cheaper computational cost St. Devi. = 0.01 means the possibility within -0.01<dx<0.01 is about 70% Design Space = [0;1] MM2 using derivative at the center IMC1 using G/H surrogate model obtained in optimization IMC2 using different G/H model by 11 samples (for st.devi.=0.01) NLMC using 3,000 CFD function calls

25. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -25- Concluding Remarks / Future Works Development of gradient/Hessian-enhanced Kriging models Application to design optimization and uncertainty analysis Direct Kriging approach is superior to indirect approach More accurate fitting on exact function Faster convergence towards global optimal design Promising inexpensive Monte-Carlo simulation at much lower cost Application to higher-dimensional / complicated design problem Robust design with inexpensive Monte-Carlo simulation Gradient/Hessian vector product-enhanced approach Thank you for your attention !!

26. Appendix

27. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -27- Moment Method Taylor series expansion by grad/Hessian at the center No information about PDF 1 st order Moment Method 2 nd order Moment Method

28. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -28- Gradient/Hessian-enhanced Kriging Implementation Details Correlation function of a RBF Estimation of hyper parameters by maximizing likelihood function with GA Correlation matrix inversion by Cholesky decomposition Search of new sample point location by maximizing Expected Improvement (EI) value with GA

29. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -29- Infill Sampling Criteria for Optimization How to find promising location on surrogate model ? Expected Improvement (EI) value Potential of being smaller than current minimum (optimal) Consider both estimated function and uncertainty (RMSE) EI-based criteria have good balance between global/local searching

30. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -30- 5D Rosenbrock Function Fitting # of pieces of information = sum of # of F/G/H net components To scatter samples is better than concentration at limited samples Approximated computational time factor G/H-enhanced surrogate model provides better performance with efficient Gradient/Hessian calculation methods

31. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -31- 1D Step Function Fitting Much better fit by G/H-enhanced direct Kriging

32. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -32- Minimization of 20D Rosenbrock Func. Minimization of 20 dimensional Rosenbrock function No computational cost for Func/Grad/Hess evaluation Expensive for construction- likelihood function maximization - inversion of correlation matrix Parallel computation for the likelihood maximization problem

33. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -33- Uncertainty Analysis Uncertainty analysis at (1.0,1.0) on 2D Rosenbrock 5 sample points for surrogate model approaches (No sample point on the center location) 2 nd order Moment Method (MM2) by G/H at the center Superior results in G/H-enhanced Inexpensive MC (IMC) St. Devi. = 0.15 means the possibility within -0.15<dx<0.15 is about 70% CDF at St. Devi.=0.15

34. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -34- Aerodynamic Data Modeling ClCl CdCd NACA0012 M=1.4 AoA=3.5[deg] Noisy in Mach number direction

35. Yamazaki, W., Dept. of Aero. Eng., Tohoku Univ. Wataru YAMAZAKI, Univ. of Wyoming -35- 2D Airfoil Shape Uncertainty Analysis Cumulative Density Function at St. Devi. of 0.01 Quadratic model only by using gradient/Hessian at optimal Additional sampling criteria to increase total model accuracy