ENGR 351 Numerical Methods Instructor: Dr. L.R. Chevalier OPTIMIZATION ENGR 351 Numerical Methods Instructor: Dr. L.R. Chevalier

Recall, when determining the root, we were seeking x where f(x) = 0

With optimization, however we are seeking f '(x) = 0

The maximum occurs when f "(x)<0

The minimum occurs when f "(x)>0

Optimization In some techniques, we determine the optima by solving the root problem f '(x) =0 If f ’(x) is not available analytically, we may use a finite difference approximation to estimate the derivative

Examples of Optimization Problems Design structures for minimum cost Design water resource project to mitigate flood damage while yielding maximum hydropower Design pump and heat transfer equipment for maximum efficiency Inventory control Optimize planning and scheduling

Methods Presented One-dimensional Unconstrained Optimization Golden Search Method Constrained Optimization Graphically Using Excel

Specific Study Objectives Understand why and where optimization occurs in engineering problem solving Understand the major elements of the general optimization problem: objective function, decision variables, and constraints

Specific Study Objectives Be able to distinguish between linear and nonlinear optimization, and between constrained and unconstrained problems Be able to define the golden ratio and understand how it makes 1-D optimization efficient Be able to solve a 2-D linear programming problem graphically

Mathematical Background An optimization or mathematical programming problem is generally stated as: find x which minimizes or maximizes f(x) subject to di(x)  ai i = 1,2,……m ei(x) = bi i=1,2,……p

Mathematical Background find x which minimizes or maximizes f(x) subject to di(x)  ai i = 1,2,……m ei(x) = bi i=1,2,……p x is the design vector (n-dimensions) f(x) is the objective function

Mathematical Background find x which minimizes or maximizes f(x) subject to di(x)  ai i = 1,2,……m ei(x) = bi i=1,2,……p di(x) are inequality constraints ei(x) are equality constraints

Mathematical Background If f(x) and the constraints are linear, we have linear programming If f(x) is quadratic, and the constraints are linear, we have quadratic programming If f(x) in not linear or quadratic, and/or the constraints are nonlinear, we have nonlinear programming

Mathematical Background find x which minimizes or maximizes f(x) subject to di(x)  ai i = 1,2,……m ei(x) = bi i=1,2,……p Without these, we have unconstrained optimization

Mathematical Background find x which minimizes or maximizes f(x) subject to di(x)  ai i = 1,2,……m ei(x) = bi i=1,2,……p With them, we have constrained optimization

1-D Unconstrained Optimization Here we see a multimodal case… however we want the global max or min! global maximum f(x) local maximum x local minimum global minimum

1-D Unconstrained Optimization We will consider the Golden Section Search method which is based on the Golden Ratio

Golden Ratio and Fibonacci Numbers The Parthenon 5th century BC This proportion was considered aesthetically pleasing by the Greeks O.61803 1

Golden Ratio and Fibonacci Numbers The Golden Ratio is related to an important mathematical series known as the Fibonacci numbers 0,1,1,2,3,5,8,13,21,34….. Each number after the first two represents the sum of the preceding two. Note the ratio of consecutive numbers

Golden Ratio and Fibonacci Numbers 0,1,1,2,3,5,8,13,21,34….. 0/1=0 1/1=1 1/2=0.5 2/3=0.667 3/5==0.6 5/8=0.625 8/13=0.615 Continue and the ratio approaches the golden ratio!

Golden Ratio and Fibonacci Numbers

1-D Unconstrained Optimization: The Golden-Section Search f(x) Pick two points, xu and xl x xl xu lo = xu-xl

1-D Unconstrained Optimization: The Golden-Section Search f(x) We will now need a two new points based on the constraints l0 = l1 + l2 l1/l0 = l2/l1 x xl xu lo = xu-xl

1-D Unconstrained Optimization: The Golden-Section Search f(x) Substituting l1/(l1+l2) = l2/l1 If the reciprocal is taken, and R = l2/l1 1+R = 1/R R2 + R - 1 = 0 x xl xu lo = xu-xl

1-D Unconstrained Optimization: The Golden-Section Search f(x) R2 + R - 1 = 0 This can be solved for the positive root x xl xu lo = xu-xl

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu lo = xu-xl Evaluate the function at these points. Two results can occur.

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu d x1 x2 d

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu d x1 x2 d Here, f(x1) > f(x2)

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu d x1 x2 d Eliminate the domain to the left of x2

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu d x1 x2 d x2 becomes xl for the next round

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu d x1 x2 d If f(x1)<f(x2), eliminate points to the right of x1

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl xu d x1 x2 d Back to the first case, here the new xl is x2

1-D Unconstrained Optimization: The Golden-Section Search f(x) x xl d xl xu old x1=x2 Because of the Golden Ratio, the previous x1 becomes the current x2

1-D Unconstrained Optimization: The Golden-Section Search f(x) x d xl xu x2

1-D Unconstrained Optimization: The Golden-Section Search f(x) x d xl xu x2 Repeat this algorithm until f(x) stabilizes

1-D Unconstrained Optimization: The Golden-Section Search Find the maximum of Let’s review the spreadsheet file opt-a.xls

Example Perform three iterations of the golden section search to maximize f(x) = -1.5x6 - 2x4 +12x using the initial guesses xl=0 and xu =2

Solution Reference opt-a.xls

Use of Solver If SOLVER is not under Tools, you’ll have to add it Use <TOOLS - ADD INS> command Choose SOLVER ADD-IN If not available as an option, you will need to install it from the original MS Office CD Reference opt-a.xls

Constrained Optimization Linear programming (LP) is an optimization approach that deals with meeting a desired objective - maximizing profit - minimizing cost Both the objective function and the constraints are linear in this case

Constrained Optimization Objective function Maximize Z = c1x1 +c2x2 +…..cnxn or Minimize Z = c1x1 +c2x2 +…..cnxn where ci = payoff of each unit of the jth activity xi = magnitude of the jth activity

Constrained Optimization Objective function Maximize Z = c1x1 +c2x2 +…..cnxn or Minimize Z = c1x1 +c2x2 +…..cnxn where ci = payoff of each unit of the jth activity xi = magnitude of the jth activity Hence, Z is the total payoff due to the total number of activities, n

Constrained Optimization The constraints can be represented by: ai1x1 +bi2x2+…..ainxn  bi where aij = amount of the ith resource that is consumed for each unit of the jth activity bi = amount of the ith resource available

Constrained Optimization Finally, we add the constraint that all activities must have a positive value xi  0

Setting up the general problem Gas processing plant that receives a fixed amount of raw gas each week Capable of processing two grades of heating gas (regular and premium) High demand for the product (I.e. guaranteed to sell) Each grade yields a different profit Similar to Problem 15.1 p. 377

Setting up the general problem Each grade has different production time and on-site storage constraints Facility is only open 120hrs/week Using the factors in the table on the next page, develop a linear programming formulation to maximize profits for this operation.

Parameters Note: a metric ton, or tonne, is equal to 1000 kg)

Parameters Let x1 = amount of regular and x2 = amount of premium

Objective Function Total Profit = 150 x1 + 175 x2 Maximize Z = 150 x1 + 175 x2

Objective Function Objective function Total Profit = 150 x1 + 175 x2 Maximize Z = 150 x1 + 175 x2 Objective function

Constraints 7x1 + 11x2  77 (material constraint) 10x1 + 8x2  120 (time constraint) x1  9 (storage constraint) x2  6 (storage constraint) x1,x2  0 (positivity constraint)

Graphical Solution

Graphical Solution Now we need to add the objective function to the plot. Start with Z = 0 (0=150x1 + 175x2) and Z = 500 (500=150x1 + 175x2)

Graphical Solution Z=1550 Still in feasible region x1 8 x2 2

Excel Solution: Using Solver

Solver Parameters Note: See Example 15.3 p. 388

Solver Solution Recall graphical solution x1 8 x2 2

Example Develop the equations (objective function and constraints) needed to optimize the problem on the next slide.

Example A construction site requires a minimum of 10,000 yd3 of sand and gravel mixture. The mixture must contain no less than 5000 yd3 of sand and no more than 6000 yd3 of gravel. The material may be obtained from two sites

Excel Solution

Excel Solution