1. One Dimensional Search
Chapter 5
One Dimensional Search
Chapter 5

2. Chapter 5 Unidimensional Search
If have a search direction, want to minimize in that
direction by numerical methods
Chapter 5
Search Methods in General
2.1. Non Sequential – Simultaneous evaluation of f at n points – no good (unless on parallel computer).
2.2. Sequential – One evaluation follows the other.

3. Chapter 5 Types of search that are better or best is often
problem dependent. Some of the types are:
a. Newton, Quasi-Newton, and Secant methods.
b. Region Elimination Methods (Fibonacci, Golden
Section, etc.).
c. Polynomial Approximation (Quadratic Interpolation,
etc.).
d. Random Search
Most methods assume
(a) a unimodal function, (b) that the min is
bracketed at the start and (c) also you start in a
direction that reduces f.
Chapter 5

4. To Bracket the Minimum
Chapter 5

5. Chapter 5

6. Chapter 5 1. Newton’s Method Newton’s method for an equation is
Application to Minimization
The necessary condition for f(x) to have a local minimum
is f′(x) = 0. Apply Newton’s method.

7. Examples
Minimize
Chapter 5
Minimize

8. Chapter 5 Advantages of Newton’s Method
(1) Locally quadratically convergent (as long as f′(x) is
positive – for a minimum).
For a quadratic function, get min in one step.
Disadvantages
Need to calculate both f′(x) and f″(x)
If f″(x)→0, method converges slowly
If function has multiple extrema, may not converge
to global optimum.
Chapter 5

9. Chapter 5 2. Finite-Difference Newton Method
Replace derivatives with finite differences
Chapter 5
Disadvantage
Now need additional function evals (3 here vs. 2 for Newton)

10. Chapter 5 3. Secant(Quasi-Newton) Method Analogous equation to (A) is
The secant approximates f″(x) as a straight line
Chapter 5

11. Chapter 5 Start the Secant method by using 2 points spanning x at
which first derivatives are of opposite sign.
For next stage, retain either x(q) or x(p) so that the pair of
derivatives still have opposite sign.
Chapter 5

12. Chapter 5 Order of Convergence
Can be expressed in various ways. Want to consider how
Chapter 5
usually slow in practice

13. Chapter 5 Fastest in practice If p = 2, quadratic convergence
Usually fast in practice
Some methods can show theoretically what
the order is.

14. Chapter 5

15. Chapter 5

16. Chapter 5

17. Chapter 5

18. Chapter 5

19. Chapter 5 Quadratic Interpolation
Approximate f(x) by a quadratic function.
Use 3 points
Chapter 5

20. Chapter 5
(or use Gaussian elimination)

21. Chapter 5

22. Chapter 5

23. Chapter 5

24. Chapter 5

25. Chapter 5

26. Chapter 5

27. Chapter 5

28. Chapter 5