Department of Mechanical Engineering, The Ohio State University Sl. #1GATEWAY Optimization

Department of Mechanical Engineering, The Ohio State University Sl. #2GATEWAY Types of Design Optimization 1.Conceptual: “Invent” several ways of doing something and pick the best. 2.Trial and Error: Make several different designs and vary the design parameters until an acceptable solution is obtained. Rarely yields the best solution. 3.Mathematical: Find the minimum mathematically.

Department of Mechanical Engineering, The Ohio State University Sl. #3GATEWAY Terms in Mathematical Optimization 1.Objective function – mathematical function which is optimized by changing the values of the design variables. 2.Design Variables – Those variables which we, as designers, can change. 3.Constraints – Functions of the design variables which establish limits in individual variables or combinations of design variables.

Department of Mechanical Engineering, The Ohio State University Sl. #4GATEWAY Steps in the Optimization Process 1.Identify the quantity or function, U, to be optimized. 2.Identify the design variables: x 1, x 2, x 3, …,x n. 3.Identify the constraints if any exist a. Equalities b. Inequalities 4. Adjust the design variables (x’s) until U is optimized and all of the constraints are satisfied.

Department of Mechanical Engineering, The Ohio State University Sl. #5GATEWAY Local and Global Optimum Designs 1.Objective functions may be unimodal or multimodal. a.Unimodal – only one optimum b.Multimodal – more than one optimum 2.Most search schemes are based on the assumption of a unimodal surface. The optimum determined in such cases is called a local optimum design. 3.The global optimum is the best of all local optimum designs.

Department of Mechanical Engineering, The Ohio State University Sl. #6GATEWAY The Objective Function, U 1.Given any feasible set of design variables, it must be possible to evaluate U. Feasible design variables are those which satisfy all of the constraints. 2.U may be simple or complex. Generally find minimum of the objective function. If the maximum is desired, then find the minimum of the objective function times –1. Max (U)  min(-U)

Department of Mechanical Engineering, The Ohio State University Sl. #7GATEWAY Example of an Objective Function x1x1 x2x2

Department of Mechanical Engineering, The Ohio State University Sl. #8GATEWAY Multimodal Objective Function local maxsaddle point

Department of Mechanical Engineering, The Ohio State University Sl. #9GATEWAY Inequality or regional constraints 1. Form: 2. p > 0 3. Divide the design space into feasible and non-feasible regions. Here the design space is the space defined by the design variables.

Department of Mechanical Engineering, The Ohio State University Sl. #10GATEWAY Equality or functional constraints 1. Form: 2. For an optimization problem, m < n 3. Often arise from physical properties or laws

Department of Mechanical Engineering, The Ohio State University Sl. #11GATEWAY Example with only inequality constraints

Department of Mechanical Engineering, The Ohio State University Sl. #12GATEWAY Example with an Equality Constraint

Department of Mechanical Engineering, The Ohio State University Sl. #13GATEWAY Example with Multiple Equality Constraints

Department of Mechanical Engineering, The Ohio State University Sl. #14GATEWAY Constrained Design Region

Department of Mechanical Engineering, The Ohio State University Sl. #15GATEWAY Approaches to Mathematical Optimization 1.Analytical methods – U is a relatively simple, closed-form analytical expression. 2.Linear Programming methods – U,  ’s, and  ’s are all linear in x’s. 3.Nonlinear searches – U,  ’s, or  ’s are nonlinear and complicated in x’s.

Department of Mechanical Engineering, The Ohio State University Sl. #16GATEWAY Analytical Methods – One Design Variable 1. There can be no equality constraints on x, since this would make the problem deterministic. 2. Inequality constraints are possible. 3. If U = U(x), the optimum occurs when 4. Must also check boundaries when inequality constraints are involved.

Department of Mechanical Engineering, The Ohio State University Sl. #17GATEWAY cost of heat loss during operation: Example – One Design Variable Insulation problem: cost of insulation of thickness x: total cost of operation: where a through d are known constants for a minimum cost: so:

Department of Mechanical Engineering, The Ohio State University Sl. #18GATEWAY Analytical Methods – Several Variables, No Constraints 2.At an optimum point, 1.U = U(x 1,x 2,x 3,…,x n ) must be nonlinear. 3.This gives n equations in n unknowns, which can be solved using some nonlinear solution procedure such as Newton’s method. 4.There are analytical tests for maximum and minimum values involving the Jacobian matrix for U, but it is usually easier to determine this by direct inspection. 5.Saddle points can be a problem.

Department of Mechanical Engineering, The Ohio State University Sl. #19GATEWAY Analytical Methods – Several Variables and Equality Constraints 1.Given: 2. Method 1: a)Solve for one of the x’s in the G equations and eliminate that variable in U. b)Optimize U with the reduced set of the design variables. c)Example:

Department of Mechanical Engineering, The Ohio State University Sl. #20GATEWAY Method 2 – Lagrange Multipliers 1.Given: 2.Used when G’s are not used to eliminate variables from U 3.Procedure: a) Introduce p new variables i such that a new objective function is formed. b)Differentiate F as if no constraints are involved.

Department of Mechanical Engineering, The Ohio State University Sl. #21GATEWAY Method 2 – Lagrange Multipliers cont. c)Solve n+p equations in n+p unknowns (x’s and ’s). n equations from p equations from G’s 4. Generally the ’s are of no direct interest if only equality constraints are present.

Department of Mechanical Engineering, The Ohio State University Sl. #22GATEWAY Example – Lagrange Multipliers Given: 1. Form F: 2. Optimize F:

Department of Mechanical Engineering, The Ohio State University Sl. #23GATEWAY Example – cont. then and

Department of Mechanical Engineering, The Ohio State University Sl. #24GATEWAY Linear Programming Given: U(x 1,x 2,…,x n ) is linear  i (x 1,x 2,…,x n ) = 0 i = 1,2,…,p are linear  I (x 1,x 2,…,x n ) > 0 i = 1,2,…,m are linear 1.No finite optimum exists unless constraints are present. 2.The optimum will occur at one of the vertices of the constraint boundaries. 3.Procedure is to start at one vertex and check vertices in a systematic manner (simplex method).

Department of Mechanical Engineering, The Ohio State University Sl. #25GATEWAY Linear Programming Example Given: Subject to the following constraints: First find the vertices by combining equations and eliminating vertices that don’t comply to all the constraints: vertexpointz x1x1 x2x /314/332/3 5010/320/3

Department of Mechanical Engineering, The Ohio State University Sl. #26GATEWAY Linear Programming Objective function contour lines – colored lines  i = inequality constraints F i = vertices

Department of Mechanical Engineering, The Ohio State University Sl. #27GATEWAY Direct Search Methods – One Design Variable Given: U(x), a<x<b 1.Vary x to optimize U(x), 2.Want to minimize number of function evaluations (number of times that U(x) is computed).

Department of Mechanical Engineering, The Ohio State University Sl. #28GATEWAY Method 1: Exhaustive Search 1.Divide the range (b-a) into equal segments and compute U at each point. 2.Pick x for the minimum U. Note that if it is desired to make function evaluations at only n interior points, then the spacing between points,  x, will be

Department of Mechanical Engineering, The Ohio State University Sl. #29GATEWAY Exhaustive search example Given: f(  ) = 7   +35 Find the minimum over the interval (0,5) using 10 interior points point  f

Department of Mechanical Engineering, The Ohio State University Sl. #30GATEWAY Method 2: Random Search 1.Objective function is evaluated at numerous randomly selected points between a and b. 2.Choose new interval about the best point. 3.Repeat the procedure until the optimum is established. Points chosen:

Department of Mechanical Engineering, The Ohio State University Sl. #31GATEWAY Random search example Given: f(  ) = 7   +35 Find the minimum on the interval (0,5) using 10 interior points point  f

Department of Mechanical Engineering, The Ohio State University Sl. #32GATEWAY Method 3: Interval Halving 1. Divide the interval into 4 equal sections, resulting in 5 points. 2. Bound the minimum and use the IOU as the new interval. 3. Repeat until the desired accuracy is reached. Determining the IOU: Case 1: if f(  2 ) < f(  3 ), then IOU is from  1 to  3. Case 2: if f(  4 ) < f(  3 ), then IOU is from  3 to  5. Case 3: otherwise IOU is from  2 to  4.

Department of Mechanical Engineering, The Ohio State University Sl. #33GATEWAY Interval Halving Example Given: f(  ) = 7   +35 Find the minimum on the interval (0,5) using 9 interior points loopitotal evaluations ii fifi ii fifi ii fifi ii fifi

Department of Mechanical Engineering, The Ohio State University Sl. #34GATEWAY Method 4: Golden Section Search 1.Divide the interval such that 2.Evaluate at  4 -z 2 and  1 +z 2. 3.Choose smallest U and reject region beyond large U. 4.Subdivide new region by the same ratio. 5.Each time there is a function evaluation, the region is reduced to times the previous size.

Department of Mechanical Engineering, The Ohio State University Sl. #35GATEWAY Golden Section Search Example Given: f(  ) = 7   +35 Find the minimum on the interval (0,5) using 10 interior points loopitotal evaluations ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi

Department of Mechanical Engineering, The Ohio State University Sl. #36GATEWAY Method 5 – Parabolic search 1.Successively approximate the shape of U as a parabola. 2.Make three function evaluations, pass the parabola through the three points, and find the minimum of the parabola. 3.Keep three best points and repeat the procedure until the optimum is established. Method 5: Parabolic Search

Department of Mechanical Engineering, The Ohio State University Sl. #37GATEWAY Method 5: Parabolic Search, cont. Need to find the parabola that fits 3 data points. This most easily accomplished by writing the parabola as follows: Now or

Department of Mechanical Engineering, The Ohio State University Sl. #38GATEWAY Parabolic Search Example Given: f(  ) = 7   cos(1.4  ) Find the minimum on the interval (0,5) using 10 interior points loopitotal evaluations ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi ii fifi

Department of Mechanical Engineering, The Ohio State University Sl. #39GATEWAY Comparison of the Direct Search Methods Method# of function evaluations Best estimate of optimum ErrorInterval of uncertainty Exhaustive Random Interval halving Golden section Iterative parabolic e-8

Department of Mechanical Engineering, The Ohio State University Sl. #40GATEWAY Optimization of Nonlinear Multivariable Systems 1.Indirect or gradient based methods - must be available. 2.Direct search methods – vary the x’s to maximize or minimize f directly.

Department of Mechanical Engineering, The Ohio State University Sl. #41GATEWAY Multivariable Optimization Searches A.Steepest descent procedure B.Optimum steepest descent procedure C.Fletcher-Powell procedure D.Powell’s method Procedures covered: I. Non-gradient methods: A.Exhaustive (Grid) search B.Random search C.Box search D.Powell’s method II. Gradient methods:

Department of Mechanical Engineering, The Ohio State University Sl. #42GATEWAY Method 1 – Grid Search 1.Divide the range for each design variable into equal segments and compute U at each point. 2.Pick x for the minimum U.

Department of Mechanical Engineering, The Ohio State University Sl. #43GATEWAY Grid Search Example Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #44GATEWAY Method 2 – Random Search 1.Objective function is evaluated at numerous randomly selected points between a and b. 2.Choose new interval about the best point. 3.Repeat the procedure until the optimum is established. Points chosen:

Department of Mechanical Engineering, The Ohio State University Sl. #45GATEWAY Random Search Example Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #46GATEWAY Method 3 – Box Method 1. Randomly choose 2n points. 2. Identify the worst point. 3. Compute the centroid of the remaining points. 4. Reflect the rejected vertex an amount  d through the centroid. 5. If the new vertex violates constraints or is worse than the rejected point, move it closer to the centroid. 6. Repeat until the optimum is found.

Department of Mechanical Engineering, The Ohio State University Sl. #47GATEWAY Box Method Example Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #48GATEWAY Powell’s method 1. Starting point: Next point: 2. Optimize along each search direction 3. Compute which search direction causes the greatest reduction of the objection function using: 4. Calculate the proposed new search direction: 5. Determine the cost of the objective function at the test point.

Department of Mechanical Engineering, The Ohio State University Sl. #49GATEWAY Powell’s method, cont. 6.Test to see if the new search direction is good using: Condition 1: Condition 2: If both conditions are true, then  is a good search direction and will replace the previous best search direction, x m. 7.Go to step 2 and repeat procedure until the optimum is found.

Department of Mechanical Engineering, The Ohio State University Sl. #50GATEWAY Example of Powell’s Method Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #51GATEWAY Gradient Based Methods Next point: Search direction:so that Minimization along the search direction: At the minimum along d (k) :

Department of Mechanical Engineering, The Ohio State University Sl. #52GATEWAY Penalty Functions 1.Used to convert constrained optimization into unconstrained optimization. 2.Combine constraint functions (  ’s and  ’s) and objective function (f) to form a new objective function. Example forms for functions G and R Where: for a, b, z are constants

Department of Mechanical Engineering, The Ohio State University Sl. #53GATEWAY Steepest Descent 1.Starting point: 2.Next point: 3.Direction to the next point is based on the gradient 4.Move a fixed distance  along d (k) and repeat procedure. Stopping procedure: If either of the following are true then the minimum has been found

Department of Mechanical Engineering, The Ohio State University Sl. #54GATEWAY Steepest Descent Example Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #55GATEWAY Optimum Steepest Descent 1.Starting point: 2.Next point: 3.Find the initial direction to the next point based on the gradient 4.Optimize along gradient 5.New search direction is perpendicular to old 6.Repeat steps 4 and 5 until the optimum is obtained

Department of Mechanical Engineering, The Ohio State University Sl. #56GATEWAY Optimum Steepest Descent Example Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #57GATEWAY Conjugate Gradient Method 1.Starting point: Next point: 2.Find the initial search direction based on the gradient 3.Check to see if If it is, then stop. Otherwise go to step 5. 4.Calculate the new search direction as where 5.Check to see if If it is, then stop.

Department of Mechanical Engineering, The Ohio State University Sl. #58GATEWAY Conjugate Gradient Method, cont. 6.Compute  k to optimize 7.Set k=k+1, let and go to step 4.

Department of Mechanical Engineering, The Ohio State University Sl. #59GATEWAY Conjugate Gradient Example Given: f(x 1,x 2 ) = 2sin(1.47  x 1 ) sin(0.34  x 2 ) + sin(  x 1 ) sin(1.9  x 2 ) Find the minimum when x 1 is allowed to vary from 0.5 to 1.5 and x 2 is allowed to vary from 0 to 2.

Department of Mechanical Engineering, The Ohio State University Sl. #60GATEWAY Credits  This module is intended as a supplement to design classes in mechanical engineering. It was developed at The Ohio State University under the NSF sponsored Gateway Coalition (grant EEC ). Contributing members include:  Gary Kinzel …………………………………….. Project supervisors  Gary Kinzel…..…………………...……………...Primary authors  Matt Detrick ……………………..……….…….. Module revisions  L. Pham…………………………………………..Speaker Based on Dr. Kinzel’s Class notes Arora, Jasbir S., Introduction to Optimum Design, Mcgraw-Hill, Inc. New York, References:

Department of Mechanical Engineering, The Ohio State University Sl. #61GATEWAY Disclaimer This information is provided “as is” for general educational purposes; it can change over time and should be interpreted with regards to this particular circumstance. While much effort is made to provide complete information, Ohio State University and Gateway do not guarantee the accuracy and reliability of any information contained or displayed in the presentation. We disclaim any warranty, expressed or implied, including the warranties of fitness for a particular purpose. We do not assume any legal liability or responsibility for the accuracy, completeness, reliability, timeliness or usefulness of any information, or processes disclosed. Nor will Ohio State University or Gateway be held liable for any improper or incorrect use of the information described and/or contain herein and assumes no responsibility for anyone’s use of the information. Reference to any specific commercial product, process, or service by trade name, trademark, manufacture, or otherwise does not necessarily constitute or imply its endorsement.