1. Design Optimization School of Engineering University of Bradford 1 Numerical optimization techniques Unconstrained multi-parameter optimization techniques Direct search (no information on derivatives used): Hooke-Jeeves’ pattern search Nelder-Mead’s sequential simplex method Powell's conjugate directions method various evolutionary techniques OPTIMIZATION TECHNIQUES

2. Design Optimization School of Engineering University of Bradford 2 Numerical optimization techniques Unconstrained multi-parameter optimization techniques Gradient-based methods (information on derivatives is used): Steepest Descent Fletcher-Reeves' Conjugate Gradient method Second order methods (information on the second derivatives is used): Newton's Method Quasi-Newton Method (constructs an approximation of the matrix of second derivatives OPTIMIZATION TECHNIQUES

3. Design Optimization School of Engineering University of Bradford 3 Numerical optimization techniques. Example 1. OPTIMIZATION TECHNIQUES Unconstrained multi- parameter optimization techniques. Difficulties in typical problems. Rosenbrock’s “banana” function F=100(x 2 -x 1 2 ) 2 +(x 1 -1) 2

4. Design Optimization School of Engineering University of Bradford 4 Numerical optimization techniques. Example 1. Nelder-Mead’s sequential simplex method OPTIMIZATION TECHNIQUES

5. Design Optimization School of Engineering University of Bradford 5 Numerical optimization techniques. Example 1. Steepest Descent OPTIMIZATION TECHNIQUES

6. Design Optimization School of Engineering University of Bradford 6 OPTIMIZATION TECHNIQUES Effect of scaling

7. Design Optimization School of Engineering University of Bradford 7 Numerical optimization techniques. Example 1. Powell's conjugate directions method OPTIMIZATION TECHNIQUES

8. Design Optimization School of Engineering University of Bradford 8 Numerical optimization techniques. Example 1. Fletcher-Reeves' Conjugate Gradient method OPTIMIZATION TECHNIQUES

9. Design Optimization School of Engineering University of Bradford 9 Numerical optimization techniques. Example 1. Newton's Method OPTIMIZATION TECHNIQUES

10. Design Optimization School of Engineering University of Bradford 10 Numerical optimization techniques. Example 1. Quasi-Newton Method OPTIMIZATION TECHNIQUES

11. Design Optimization School of Engineering University of Bradford 11 Numerical optimization techniques OPTIMIZATION TECHNIQUES Unconstrained multi- parameter optimization techniques. Example 2.

12. Design Optimization School of Engineering University of Bradford 12 Numerical optimization techniques OPTIMIZATION TECHNIQUES Example 2. Newton, trajectories of search from three starting points

13. Design Optimization School of Engineering University of Bradford 13 Numerical optimization techniques OPTIMIZATION TECHNIQUES Example 2. Newton, trajectory of search from the first starting point. First iteration.

14. Design Optimization School of Engineering University of Bradford 14 Numerical optimization techniques OPTIMIZATION TECHNIQUES Example 2. Newton, trajectory of search from the first starting point. Second iteration, optimization stops at the saddle point

15. Design Optimization School of Engineering University of Bradford 15 Numerical optimization techniques OPTIMIZATION TECHNIQUES Example 2. Newton, trajectory of search from the third starting point. First iteration.

16. Design Optimization School of Engineering University of Bradford 16 Numerical optimization techniques OPTIMIZATION TECHNIQUES Example 2. Newton, trajectory of search from the third starting point. Second iteration.

17. Design Optimization School of Engineering University of Bradford 17 Numerical optimization techniques Penalty function approaches Exterior penalty function OPTIMIZATION TECHNIQUES

18. Design Optimization School of Engineering University of Bradford 18 Numerical optimization techniques OPTIMIZATION TECHNIQUES Exterior penalty function approach. First iteration

19. Design Optimization School of Engineering University of Bradford 19 Numerical optimization techniques OPTIMIZATION TECHNIQUES Exterior penalty function approach. Second iteration

20. Design Optimization School of Engineering University of Bradford 20 Numerical optimization techniques OPTIMIZATION TECHNIQUES Exterior penalty function approach. Third iteration

21. Design Optimization School of Engineering University of Bradford 21 Numerical optimization techniques Penalty function approaches Interior penalty function OPTIMIZATION TECHNIQUES

22. Design Optimization School of Engineering University of Bradford 22 Numerical optimization techniques OPTIMIZATION TECHNIQUES Interior penalty function approach. First iteration

23. Design Optimization School of Engineering University of Bradford 23 Numerical optimization techniques OPTIMIZATION TECHNIQUES Interior penalty function approach. Second iteration

24. Design Optimization School of Engineering University of Bradford 24 Numerical optimization techniques OPTIMIZATION TECHNIQUES Interior penalty function approach. Third iteration