1. L5 Optimal Design concepts pt A
Homework
Review
“Minimum” definitions
Weierstrass Theorem
Gradient vector, Hessian matrix
Taylor Series Expansions
Summary
Test 1 Wed
formula sheet

2. Graphical Solution Sketch coordinate system Plot constraints
Determine feasible region
Plot f(x) contours
Find opt solution x* & opt value f(x*)

3. 5. Find Optimal solution & value
Opt. solution
point D
x*= [4,12]
Opt. Value
P=4(400)+12(600)
P=8800
f(x*)=8800
Figure 3.5 Graphical solution to the profit maximization problem: optimum point D = (4, 12); maximum profit, P = 8800.

4. Optimization Methods
Figure 4.1 Classification of optimization methods.

5. Global/local optima Global Maximum? Local Maximum? Global Minimum?
Local Minimum?
Global = absolute
Local = relative
Figure 4.2 Representation of optimum points. (a) The unbounded domain and function (no global optimum). (b) The bounded domain and function (global minimum and maximum exist).
Closed & Bounded

6. Global (Absolute) Min
x* is a global minimum of f(x) IFF f(x*)≤ f(x) for all x in the feasible set Note that
“Strong (strict)” versus “weak” minima
f(x*)< f(x)
f(x*)≤ f(x)

7. Local (Relative) Min
x* is a global minimum of f(x) IFF f(x*)≤ f(x) For all x in a small neighborhood N of x* in the feasible set Where N is defined as:

8. Global/local optima Global Maximum? f(x*)≤ f(x) Anywhere in S
Local Maximum?
In small neighborhood N
Figure 4.2 Representation of optimum points. (a) The unbounded domain and function (no global optimum). (b) The bounded domain and function (global minimum and maximum exist).
Closed & Bounded

9. Unconstrained Minimum
Figure 4.3 Representation of unconstrained minimum for Example 4.1.

10. Constrained Minimum
Figure 4.4 Representation of constrained minimum for Example 4.2.

11. Existence of a Global Minimum
Weierstrass Theorem: 1. If f(x) is continuous 2. On non-empty set S and 3. S is closed and bounded Then f(x) has a global minimum in S recall open interval (1,3) versus closed [1,3]

12. NOTE: If Weierstrass conditions are not satisfied
A global minimum may still exist! The power of the theorem is that it guarantees a global min if conditions are satisfied. e.g. f(x)= x2 on open interval

13. Gradient is ⊥ to tangent plane
Figure 4.5 Gradient vector for f(x1, x2, x3) at the point x*.

14. Gradient example
1.6
1.2
Figure 4.6 Gradient vector (that is not to scale) for the function f(x) of Example 4.5 at the point (1.8, 1.6).

15. Second Partial Deriivatives of a function f(x)
Hessian Matrix
What does the x* mean?

16. Hessian Example

17. Taylor Series Expansion
Assume f(x) is:
1. Continuous function of a single variable x
2. Differentiable n times
3. x ∈ S, where S is non-empty, closed, and bounded
4. therefore x* is a possible optima

18. Taylor Series Approximations

19. Taylor Series – Mult. Variables
Estimate f(x) using Taylor Series
Let

20. Change in function
Let’s define a change in the function with the symbol ∆
∆-Delta ∇-Del
Let’s define a first order change in the function

21. Taylor Exp of MV

22. Summary “Minimum” definitions Weierstrass Theorem
Gradient vector, Hessian matrix
Taylor Series Expansions