L25 Numerical Methods part 5 Project Questions Homework Review Tips and Tricks Summary 1

H24 2 F F

10.57 revisited 3

Prob

Conjugate Gradient 7 Proof:

“Deflected” Steepest Descent 8 A comparison of the convergence of gradient descent with optimal step size (in green) and conjugate vector (in red) for minimizing a quadratic function associated with a given linear system. Conjugate gradient, assuming exact arithmetic, converges in at most n steps where n is the size of the matrix of the system (here n=2). Wik gradient descent ient_method

Higher Order Methods 9

Optimization Project Formulating Computer Modeling Solving/executing Evaluating (your “solution”) Analyzing the sensitivity of your solution 10

1. Tips: Formulating 11 Functional requirements (HoQ) Eng. Characteristics (i.e. quantifiable measures) Identify design variables, names, symbols, units, limits Develop Objective function Retrieve or develop analytical formulas/models Develop constraints (laws of nature, man & economics) “Principal of Optimum Sloppiness”, significant figures?

2. Tips: Computer Modeling Pre-test custom-written code Hand-check (w/calculator): f(x), g(x), h(x) at some x(1) Eliminate ratios, if possible (to avoid divide by zero) Eliminate non-differentiable functions (such as abs(), max()) Check analytical derivatives w/FD derivatives Exploit available library routines Scale variables and or constraints if difficulties arise 12

3. Tips: Solving/Executing 13 Test “optimizer” w/ known problems/solutions Solve from multiple starting points If algorithm fails, monitor each iteration Record statistics: constraint values, solutions

4. Tips: Evaluating the Solution 14 Hand-check (calculator): f(x*), g(x*), h(x*) Evaluate constraint activity – Violated – Non-binding/inactive – Binding/active Do results make physical sense?

5. Tips: Analyzing the sensitivity 15 Relax R.H.S. Record Δf(x) for Δx Change cost coefficients in f(x) Vary parameters in g(x), h(x) Remember: A, b and c’s Look for opportunity!

Test 5 on Wed T/F Region elimination methods Steepest descent algorithim Conjugate Gradient algorithm Be prepared to do hand calculations. 16

Alternate Equal Interval Golden Section Equal Interval aka “Exhaustive” Fractional Reduction 17 Add these formulas to your notes for next test!

Summary Steepest descent algorithm may stall Conjugate Gradient Convergence in n iterations (n=# of design var’s) Still has lots of Fcn evals (in line search) May need to restart after n+1 iterations Use “TIPS” to facilitate your project 18