Page 1 Page 1 Engineering Optimization Second Edition Authors: A. Rabindran, K. M. Ragsdell, and G. V. Reklaitis Chapter-2 (Functions of a Single Variable) Presenter: Avishek Nag June 11, 2010
Page 2 Page 2 Dedicated to… All the neo-graduates whose insatiable urge to learn has not yet dimmed
Page 3 Page 3 What is a Function? Is a rule that assigns to every choice of x a unique value y =ƒ(x). Domain of a function is the set of all possible input values (usually x), which allows the function formula to work. Range is the set of all possible output values (usually y), which result from using the function formula.
Page 4 Page 4 Unconstrained and constrained function –Unconstrained: when domain is the entire set of real numbers R –Constrained: domain is a proper subset of R Continuous, discontinuous and discrete What is a Function?
Page 5 Page 5 Monotonic and unimodal functions –Monotonic: –Unimodal: What is a Function? ƒ(x) is unimodal on the interval if and only if it is monotonic on either side of the single optimal point x* in the interval. Unimodality is an extremely important functional property used in optimization.
Page 6 Page 6 A monotonic increasing function A monotonic decreasing function An unimodal function
Page 7 Page 7 Optimality Criteria In considering optimization problems, two questions generally must be addressed: 1.Static Question. How can one determine whether a given point x* is the optimal solution? 2.Dynamic Question. If x* is not the optimal point, then how does one go about finding a solution that is optimal?
Page 8 Page 8 Local and global optimum Optimality Criteria
Page 9 Page 9
Page 10 Page 10 Identification of Single-Variable Optima For finding local minima (maxima) Proof follows… These are necessary conditions, i.e., if they are not satisfied, x* is not a local minimum (maximum). If they are satisfied, we still have no guarantee that x* is a local minimum (maximum). AND
Page 11 Page 11 Stationary Point and Inflection Point A stationary point is a point x* at which An inflection point or saddle-point is a stationary point that does not correspond to a local optimum (minimum or maximum). To distinguish whether a stationary point is a local minimum, a local maximum, or an inflection point, we need the sufficient conditions of optimality.
Page 12 Page 12 Theorem Suppose at a point x* the first derivative is zero and the first nonzero higher order derivative is denoted by n. –If n is odd, then x* is a point of inflection. –If n is even, then x* is a local optimum. Moreover: –If that derivative is positive, then the point x* is a local minimum. –If that derivative is negative, then the point x* is a local maximum.
Page 13 Page 13 An Example Thus the first non-vanishing derivative is 3 (odd), and x = 0 is an inflection point.
Page 14 Page 14 An Example Stationary points x = 0, 1, 2, 3 -Local minimum -Local maximum At x = 0- Inflection point
Page 15 Page 15 Algorithm to Find Global Optima
Page 16 Page 16 An Example Stationary points x = -1, 3
Page 17 Page 17 Region Elimination Methods
Page 18 Page 18 Bounding Phase –An initial coarse search that will bound or bracket the optimum Interval Refinement Phase –A finite sequence of interval reductions or refinements to reduce the initial search interval to desired accuracy Region Elimination Methods
Page 19 Page 19 Bounding Phase Swann’s method –If –Else if the inequalities are reversed –If is positive is negative the minimum lies between
Page 20 Page 20 Interval Refinement Phase Interval halving
Page 21 Page 21 Interval Refinement Phase
Page 22 Page 22 Polynomial Approximation or Point-Estimation Technique Quadratic Approximation Method
Page 23 Page 23
Page 24 Page 24
Page 25 Page 25 Successive Quadratic Estimation Method
Page 26 Page 26 Methods Requiring Derivatives Newton-Raphson Method
Page 27 Page 27 Bisection Method
Page 28 Page 28 Secant Method
Page 29 Page 29 Cubic Search Method
Page 30 Page 30 Cubic Search Method
Page 31 Page 31 Summary We learned… –Functions –Optimality criteria –Identification of single variable optima Region elimination methods Polynomial approximation or point-estimation technique Methods requiring derivatives