1. ENGR 351 Numerical Methods Instructor: Dr. L.R. Chevalier
OPTIMIZATION
ENGR 351 Numerical Methods
Instructor: Dr. L.R. Chevalier

2. Recall, when determining the root, we were seeking x where f(x) = 0

3. With optimization, however we are seeking f '(x) = 0

4. The maximum occurs when f "(x)<0

5. The minimum occurs when f "(x)>0

6. Optimization
In some techniques, we determine the optima by solving the root problem f '(x) =0
If f ’(x) is not available analytically, we may use a finite difference approximation to estimate the derivative

7. Examples of Optimization Problems
Design structures for minimum cost
Design water resource project to mitigate flood damage while yielding maximum hydropower
Design pump and heat transfer equipment for maximum efficiency
Inventory control
Optimize planning and scheduling

8. Methods Presented One-dimensional Unconstrained Optimization
Golden Search Method
Constrained Optimization
Graphically
Using Excel

9. Specific Study Objectives
Understand why and where optimization occurs in engineering problem solving
Understand the major elements of the general optimization problem: objective function, decision variables, and constraints

10. Specific Study Objectives
Be able to distinguish between linear and nonlinear optimization, and between constrained and unconstrained problems
Be able to define the golden ratio and understand how it makes 1-D optimization efficient
Be able to solve a 2-D linear programming problem graphically

11. Mathematical Background
An optimization or mathematical programming problem is generally stated as:
find x which minimizes or maximizes f(x) subject to
di(x)  ai i = 1,2,……m
ei(x) = bi i=1,2,……p

12. Mathematical Background
find x which minimizes or maximizes f(x) subject to
di(x)  ai i = 1,2,……m
ei(x) = bi i=1,2,……p
x is the design vector (n-dimensions)
f(x) is the objective function

13. Mathematical Background
find x which minimizes or maximizes f(x) subject to
di(x)  ai i = 1,2,……m
ei(x) = bi i=1,2,……p
di(x) are inequality constraints
ei(x) are equality constraints

14. Mathematical Background
If f(x) and the constraints are linear, we have linear programming
If f(x) is quadratic, and the constraints are linear, we have quadratic programming
If f(x) in not linear or quadratic, and/or the constraints are nonlinear, we have nonlinear programming

15. Mathematical Background
find x which minimizes or maximizes f(x) subject to
di(x)  ai i = 1,2,……m
ei(x) = bi i=1,2,……p
Without these, we have unconstrained optimization

16. Mathematical Background
find x which minimizes or maximizes f(x) subject to
di(x)  ai i = 1,2,……m
ei(x) = bi i=1,2,……p
With them, we have constrained optimization

17. 1-D Unconstrained Optimization
Here we see a multimodal case…
however we want the global max or min!
global maximum
f(x)
local maximum x
local minimum
global minimum

18. 1-D Unconstrained Optimization
We will consider the Golden Section Search method which is based on the Golden Ratio

19. Golden Ratio and Fibonacci Numbers
The Parthenon
5th century BC
This proportion was considered
aesthetically pleasing by the Greeks
O.61803 1

20. Golden Ratio and Fibonacci Numbers
The Golden Ratio is related to an important mathematical series known as the Fibonacci numbers
0,1,1,2,3,5,8,13,21,34…..
Each number after the first two represents the sum of the preceding two. Note the ratio of consecutive numbers

21. Golden Ratio and Fibonacci Numbers
0,1,1,2,3,5,8,13,21,34…..
0/1=0
1/1=1
1/2=0.5
2/3=0.667
3/5==0.6
5/8=0.625
8/13=0.615
Continue and the ratio approaches the golden ratio!

22. Golden Ratio and Fibonacci Numbers

23. 1-D Unconstrained Optimization: The Golden-Section Search
f(x)
Pick two points,
xu and xl x xl xu
lo = xu-xl

24. 1-D Unconstrained Optimization: The Golden-Section Search
f(x)
We will now need a two new points based on the constraints
l0 = l1 + l2
l1/l0 = l2/l1 x xl xu
lo = xu-xl

25. 1-D Unconstrained Optimization: The Golden-Section Search
f(x)
Substituting
l1/(l1+l2) = l2/l1
If the reciprocal is taken, and
R = l2/l1
1+R = 1/R
R2 + R - 1 = 0 x xl xu
lo = xu-xl

26. 1-D Unconstrained Optimization: The Golden-Section Search
f(x)
R2 + R - 1 = 0
This can be solved for the positive root x xl xu
lo = xu-xl

27. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu
lo = xu-xl
Evaluate the function at these points. Two results can occur.

28. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu d x1 x2 d

29. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu d x1 x2 d
Here, f(x1) > f(x2)

30. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu d x1 x2 d
Eliminate the domain to the left of x2

31. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu d x1 x2 d
x2 becomes xl for the next round

32. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu d x1 x2 d
If f(x1)<f(x2), eliminate points to the right of x1

33. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl xu d x1 x2 d
Back to the first case, here the new xl is x2

34. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x xl d xl xu
old x1=x2
Because of the Golden Ratio, the previous x1 becomes the current x2

35. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x d xl xu x2

36. 1-D Unconstrained Optimization: The Golden-Section Search
f(x) x d xl xu x2
Repeat this algorithm until f(x) stabilizes

37. 1-D Unconstrained Optimization: The Golden-Section Search
Find the maximum of
Let’s review the spreadsheet file
opt-a.xls

38. Example
Perform three iterations of the golden section search to maximize
f(x) = -1.5x6 - 2x4 +12x
using the initial
guesses
xl=0 and xu =2

39. Solution
Reference opt-a.xls

40. Use of Solver If SOLVER is not under Tools, you’ll have to add it
Use <TOOLS - ADD INS> command
Choose SOLVER ADD-IN
If not available as an option, you will need to install it from the original MS Office CD
Reference opt-a.xls

41. Constrained Optimization
Linear programming (LP) is an optimization approach that deals with meeting a desired objective
- maximizing profit
- minimizing cost
Both the objective function and the constraints are linear in this case

42. Constrained Optimization
Objective function
Maximize Z = c1x1 +c2x2 +…..cnxn or
Minimize Z = c1x1 +c2x2 +…..cnxn
where ci = payoff of each unit of the jth activity
xi = magnitude of the jth activity

43. Constrained Optimization
Objective function
Maximize Z = c1x1 +c2x2 +…..cnxn or
Minimize Z = c1x1 +c2x2 +…..cnxn
where ci = payoff of each unit of the jth activity
xi = magnitude of the jth activity
Hence, Z is the total payoff due to the total number of activities, n

44. Constrained Optimization
The constraints can be represented by:
ai1x1 +bi2x2+…..ainxn  bi
where aij = amount of the ith resource that is consumed for each unit of the jth activity
bi = amount of the ith resource available

45. Constrained Optimization
Finally, we add the constraint that all activities must have a positive value
xi  0

46. Setting up the general problem
Gas processing plant that receives a fixed amount of raw gas each week
Capable of processing two grades of heating gas (regular and premium)
High demand for the product (I.e. guaranteed to sell)
Each grade yields a different profit
Similar to Problem 15.1 p. 377

47. Setting up the general problem
Each grade has different production time and on-site storage constraints
Facility is only open 120hrs/week
Using the factors in the table on the next page, develop a linear programming formulation to maximize profits for this operation.

48. Parameters
Note: a metric ton, or tonne, is equal to 1000 kg)

49. Parameters
Let x1 = amount of regular and
x2 = amount of premium

50. Objective Function Total Profit = 150 x1 + 175 x2
Maximize Z = 150 x x2

51. Objective Function Objective function Total Profit = 150 x1 + 175 x2
Maximize Z = 150 x x2
Objective
function

52. Constraints 7x1 + 11x2  77 (material constraint)
10x1 + 8x2  (time constraint)
x1  (storage constraint)
x2  (storage constraint)
x1,x2  (positivity constraint)

53. Graphical Solution

54. Graphical Solution
Now we need to add the objective function to the plot. Start with
Z = 0 (0=150x x2)
and
Z = 500 (500=150x x2)

55. Graphical Solution
Z=1550
Still in feasible region
x1 8
x2 2

56. Excel Solution: Using Solver

57. Solver Parameters
Note: See Example 15.3 p. 388

58. Solver Solution
Recall graphical solution
x1 8
x2 2

59. Example
Develop the equations (objective function and constraints) needed to optimize the problem on the next slide.

60. Example
A construction site requires a minimum of 10,000 yd3 of sand and gravel mixture. The mixture must contain no less than 5000 yd3 of sand and no more than 6000 yd3 of gravel. The material may be obtained from two sites

61. Excel Solution

62. Excel Solution