One Dimensional Search Chapter 5 One Dimensional Search Chapter 5
Chapter 5 Unidimensional Search If have a search direction, want to minimize in that direction by numerical methods Chapter 5 Search Methods in General 2.1. Non Sequential – Simultaneous evaluation of f at n points – no good (unless on parallel computer). 2.2. Sequential – One evaluation follows the other.
Chapter 5 Types of search that are better or best is often problem dependent. Some of the types are: a. Newton, Quasi-Newton, and Secant methods. b. Region Elimination Methods (Fibonacci, Golden Section, etc.). c. Polynomial Approximation (Quadratic Interpolation, etc.). d. Random Search Most methods assume (a) a unimodal function, (b) that the min is bracketed at the start and (c) also you start in a direction that reduces f. Chapter 5
To Bracket the Minimum Chapter 5
Chapter 5
Chapter 5 1. Newton’s Method Newton’s method for an equation is Application to Minimization The necessary condition for f(x) to have a local minimum is f′(x) = 0. Apply Newton’s method.
Examples Minimize Chapter 5 Minimize
Chapter 5 Advantages of Newton’s Method (1) Locally quadratically convergent (as long as f′(x) is positive – for a minimum). For a quadratic function, get min in one step. Disadvantages Need to calculate both f′(x) and f″(x) If f″(x)→0, method converges slowly If function has multiple extrema, may not converge to global optimum. Chapter 5
Chapter 5 2. Finite-Difference Newton Method Replace derivatives with finite differences Chapter 5 Disadvantage Now need additional function evals (3 here vs. 2 for Newton)
Chapter 5 3. Secant(Quasi-Newton) Method Analogous equation to (A) is The secant approximates f″(x) as a straight line Chapter 5
Chapter 5 Start the Secant method by using 2 points spanning x at which first derivatives are of opposite sign. For next stage, retain either x(q) or x(p) so that the pair of derivatives still have opposite sign. Chapter 5
Chapter 5 Order of Convergence Can be expressed in various ways. Want to consider how Chapter 5 usually slow in practice
Chapter 5 Fastest in practice If p = 2, quadratic convergence Usually fast in practice Some methods can show theoretically what the order is.
Chapter 5
Chapter 5
Chapter 5
Chapter 5
Chapter 5
Chapter 5 Quadratic Interpolation Approximate f(x) by a quadratic function. Use 3 points Chapter 5
Chapter 5 (or use Gaussian elimination)
Chapter 5
Chapter 5
Chapter 5
Chapter 5
Chapter 5
Chapter 5
Chapter 5
Chapter 5