1. Computational Methods for Design Lecture 4 – Introduction to Sensitivities John A. Burns C enter for O ptimal D esign A nd C ontrol I nterdisciplinary C enter for A pplied M athematics Virginia Polytechnic Institute and State University Blacksburg, Virginia 24061-0531 A Short Course in Applied Mathematics 2 February 2004 – 7 February 2004 N∞M∞T Series Two Course Canisius College, Buffalo, NY

2. A Falling Object “Newton’s Second Law”. y(t) { { AIR RESISTANCE

3. System of Differential Equations

4. Parameters IN REAL PROBLEMS THERE ARE PARAMETERS SOLUTIONS DEPEND ON THESE PARAMETERS WE WILL BE INTERESTED IN COMPUTING SENSITIVITIES WITH RESPECT TO THESE PARAMETERS

5. Examples: n=m=1 CONTINUOUS EVERYWHERE UNIQUE SOLUTION CONTINUOUS WHEN UNIQUE SOLUTION

6. Logistics Equation

7. Computing Sensitivities HOW DO WE COMPUTE THE SENSITIVITIES …

8. SEM Example 1 DIFFERENTIATE

9. SEM Example 1

10. SEM Method SOLVE THE SYSTEM (DE) – (SE) (DE) (SE) 1.WHY DO IT THIS WAY ? 2.WE DERIVED (SE) BY USING THE KNOWN SOLUTION … HOW DO WE FIND (SE) IN GENERAL? 3.HOW GENERAL IS THIS PROCESS?

11. Derivation of SEN Eq DIFFERENTIATE THE EQUATION WITH RESPECT TO q INTERCHANGE THE ORDER OF DIFFERENTIATION

12. Derivation of SEN Eq (DE) (SE)

13. SEM Method (DE) (SE)

14. Explicit Euler for SEQs t0t0 (x 0,s 0 ) R2R2

15. Explicit Euler for SEQs

16. SOLVE BOTH DE AND SE TOGETHER HOW DOES IT WORK?

17. MATLAB Code for SEM Set q Set x 0 and s 0 Set h Time interval Set ICs Explicit Euler

18. DE Solution x(t)

19. SE Solution s(t)

20. Special Structure of SE’s (DE) (SE) (DE) (SE) FIRST: SOLVE (DE) SECOND: SOLVE (SE)

21. Logistics Equation

22. SEQ for the Logistics Equation DIFFERENTIATE THE EQUATION WITH RESPECT TO q 1

23. SEQ for the Logistics Equation

25. INTERCHANGE THE ORDER OF DIFFERENTIATION

26. SEQ for the Logistics Equation

28. NEED SENSITIVITY WITH RESPECT TO q 2

29. SEQ for the Logistics Equation 2 DIFFERENTIATE THE EQUATION WITH RESPECT TO q 2

30. SEQ for the Logistics Equation 2

32. INTERCHANGE THE ORDER OF DIFFERENTIATION

33. SEQ for the Logistics Equation 2

34. SEQ’s for the Logistics Equation FROM THE FIRST PARTIAL THE LOGISTICS EQUATION

35. SEQ’s for the Logistics Equation FIRST: SOLVE (DE) SECOND: SOLVE (SEs)

36. Model Problem #1 q q q SENSITIVITY The sensitivity equation for s(x, q ) =  q w(x, q ) in the “ physical ” domain  ( q ) = (0, q ) is given by Can be made “ rigorous ” by the method of mappings. MORE ABOUT THIS NEAR THE END q q q  q,q, q q q

37. Typical Cost Function WHERE w( x, q ) USUALLY SATISFIES A DIFFERENTIAL EQUATION AND q IS A PARAMETER (OR VECTOR OF PARAMETERS) q q q q q q q q q q q q q q THE CHAIN RULE PRODUCES OR (Reality) USING NUMERICAL SOLUTIONS hh hh CONTINUOUS SENSITIVITY DISCRETE SENSITIVITY

38. Computing Gradients (I) BY FINITE DIFFERENCES q q0q0 q0q0 q0q0 q h h h TYPICAL APPROACHES TO COMPUTE q q =q0q =q0 h (II) BY DISCRETE SENSITIVITIES q0q0 q0q0 q0q0 q0q0 q0q0 h h h h

39. Computing Gradients FINITE DIFFERENCES REQUIRES 2 NON-LINEAR SOLVES IF SHAPE IS A DESIGN VARIABLE, FD REQUIRES 2 MESH GENERATIONS DISCRETE SENSITIVITIES REQUIRES THE EXISTENCE OF THE DISCRETE SENSITIVITY IF SHAPE IS A DESIGN VARIABLE, THE DISCRETE SENSITIVITY LEADS TO MESH DERIVATIVES COMPUTATIONS WHAT IS THE “ CONTINUOUS / HYBRID ” SENSITIVITY EQUATION METHOD? --- SEM q0q0 q0q0 q0q0 h h h, k APPROXIMATE

40. A Sensitivity Equation Method FOR q > 1 AND h=q / (N+1) CONSIDER (FORMAL ) h h h h h NUMERICAL APPROXIMATION x=0 x=1 x=q x w(x) w h (x) = Finite Element Approximation q q,q,  DISCRETE STATE EQUATION

41. A Sensitivity Equation Method h h h h h h h q q,q,  q q  q,q, q h  IMPORTANT OBSERVATIONS l The sensitivity equations are linear The sensitivity equation “ solver ” can be constructed independently of the forward solver -- SENSE™ When done correctly “ mesh gradients ” are not required

42. A Sensitivity Equation Method FOR q > 1 AND k = q / (M+1) CONSIDER (FORMAL) 2 nd NUMERICAL APPROXIMATION x=0 x=1 x=q x s(x)=  q w(x,q) s h,k (x) = Finite Element Approximation of  h h,k q q  q,q, q q q h h h

43. Convergence Issues q q q h h h q k h,k THEOREM. The finite element scheme is asymptotically consistent. qq h h h k k a trust region method should (might?) converge. qq h h k When the erroris small, then IDEA: J. T. Borggaard and J. A. Burns, “ A PDE Sensitivity Equation Method for Optimal Aerodynamic Design ”, Journal of Computational Physics, Vol.136 (1997), 366-384. R. G. Carter, “ On the Global Convergence of Trust-Region Algorithms Using Inexact Gradient Information ”, SIAM J. Num. Anal., Vol 28 (1991), 251-265. J. T. Borggaard, “ The Sensitivity Equation Method for Optimal Design ”, Ph.D. Thesis, Virginia Tech, Blacksburg, VA, 1995.

44. Convergence Issues N=16, M=32

45. Convergence Issues qq h hh THE CASE k = h is often used, but may not be “ good enough ” NOT CONVERGENT

46. Timing Issues THE CASE k = 2h offers flexibility and qq h h 2h2h convergence. But, what about timings? Approximately 96.6% of cpu time spent in function evaluations Approximately 02.4% of cpu time spent in gradient evaluations

47. Mathematics Impacts “ Practically ” UNDERSTANDING THE PROPER MATHEMATICAL FRAMEWORK CAN BE EXPLOITED TO PRODUCE BETTER SCIENTIFIC COMPUTING TOOLS  A REAL JET ENGINE WITH 20 DESIGN VARIABLES l PREVIOUS ENGINEERING DESIGN METHODOLOGY REQUIRED 8400 CPU HRS ~ 1 YEAR l USING A HYBRID SEM DEVELOPED AT VA TECH AS IMPLEMENTED BY AEROSOFT IN SENSE™ REDUCED THE DESIGN CYCLE TIME FROM... 8400 CPU HRS ~ 1 YEAR TO 480 CPU HRS ~ 3 WEEKS NEW MATHEMATICS WAS THE ENABLING TECHNOLOGY

48. Special Structure of SE’s (DE) (SE) (DE) (SE) FIRST: SOLVE (DE) SECOND: SOLVE (SE)

49. END