1. Engineering Optimization
Concepts and Applications
Fred van Keulen
Matthijs Langelaar
CLA H21.1

2. Optimization problem
Design variables: variables with which the design problem is parameterized:
Objective: quantity that is to be minimized (maximized) Usually denoted by: ( “cost function”)
Constraint: condition that has to be satisfied
Inequality constraint:
Equality constraint:

3. Optimization problem (cont.)
General form of optimization problem:

4. Classification Problems: Responses: Variables:
Constrained vs. unconstrained
Single level vs. multilevel
Single objective vs. multi-objective
Deterministic vs. stochastic
Responses:
Linear vs. nonlinear
Convex vs. nonconvex
Smooth vs. nonsmooth
Variables:
Continuous vs. discrete (integer, ordered, non-ordered)

5. Solving optimization problems
Optimization problems are typically solved using an iterative algorithm:
Responses
Derivatives of responses
(design sensi- tivities)
Model
Constants
Design variables
Optimizer

6. Optimization pitfalls!
Proper problem formulation critical!
Choosing the right algorithm for a given problem
Many algorithms contain lots of control parameters
Optimization tends to exploit weaknesses in models
Optimization can result in very sensitive designs
Some problems are simply too hard / large / expensive

7. Exercises Exercise 1: Introduction to the valve spring design problem
Study analysis model
Formulation of spring optimization model
Exercise 2: Model behavior / optimization formulation
Study model properties (monotonicity, convexity, nonlinearity)
Optimization problem formulation

8. Course overview
General introduction, problem formulation, design space / optimization terminology
Modeling, model simplification
Optimization of unconstrained / constrained problems
Single-variable, zeroth-order and gradient-based optimization algorithms
Design sensitivity analysis (FEM)
Topology optimization

9. Defining a design model and optimization problem
1. What can be changed and how can the design be described?
Dimensions
Stacking sequence of laminates
Ply orientation of laminates
Thicknesses
For structures: distinguish sizing, material and shape variables
Bridgestone aircraft tire

10. Defining the optimization problem
2. What is “best”? Define an objective function:
Weight
Production cost
Life-time cost
Profits
3. What are the restrictions? Define the constraints:
Stresses
Buckling load
Eigenfrequency

11. Defining the optimization problem (cont.)
4. Optimization: find a suitable algorithm to solve the optimization problem. Choice depends on problem characteristics:
Number of design variables, constraints
Computational cost of function evaluation
Sensitivities available?
Continuous / discrete design variables?
Smooth responses?
Numerical noise?
Many local optima? (nonconvex)
Nonsmooth response: for example maximum stresses in a structure (only C0). Other models can even be discontinuous.

12. Summary Defining an optimization problem:
Choose design variables and their bounds
Formulate objective (best?)
Formulate constraints (restrictions?)
Choose suitable optimization algorithm

13. Standard forms Several standard forms exist: Negative null form:
Positive null form:
Neg. unity form:
Pos. unity form:

14. Structural optimization examples
Typical objective function: weight
Typical constraint: maximum stress, maximum displacement
Note the scaling!
Scaled vs. Unscaled

15. Example: minimum weight tubular column design
Length l given
Load P given
Design variables:
Radius R  [Rmin, Rmax]
Wall thickness t  [tmin, tmax]
Objective: minimum mass
Constraints: buckling, stress R t P l

16. Tubular column design Design problem:
Definition, substitution of elementary equations, normal form.
Addition of condition associated to the modeling assumptions (thin-walled tube, A and I use this)

17. Tubular column design (2)
Alternative formulation: Ro Ri P l

18. Multi-objective problems
Minimize c(x) s.t. g(x)  0, h(x) = 0
Input from designer required! Popular approach: replace by weighted sum:
Vector!
Optimum, clearly, depends on choice of weights
Pareto optimal point: “no other feasible point exists that has a smaller ci without having a larger cj”

19. Multi-objective problems (cont.)
Examples of multi-objective problems:
Design of a structure for
Minimal weight and
Minimal stresses
Design of reduction gear unit for
Minimal volume
Maximal fatigue life
Design of a truck for
Minimal fuel 80 km/h
Minimal acceleration time for 0 – 40 km/h
Minimal acceleration time for 40 – 90 km/h
Minimal stress important for fatigue.

20. Pareto set
Pareto point: “Cannot improve an objective without worsening another” c2
Pareto set
Attainable set
Pareto point c1

21. Pareto set (cont.)
Alternative view:
Pareto set c1 c2 x

22. Pareto set (cont.) Pareto set can be disjoint: Attainable set c2

23. Hierarchical systems
Large system can be decomposed into subsystems / components:
Optimization requires specialized techniques, multilevel optimization
Biomass reactor for power production.

24. Structural hierarchical systems
Example: wing box
Too many design variables to treat at once
Global level: global loads, global dimensions
The global level simulation provides the loads for the local problems. However, changing the local properties will change the load distribution, which also affects the global level and other local problems. An iterative scheme is needed to solve this optimization problem.
Local (rib / stiffner) level: plate thickness, fiber orientation

25. Contents Defining an optimization problem
The design space & problem characteristics
Model simplification

26. The design space Design space = set of all possible designs Example:
kmax
Feasible domain F k2
The cost of the springs is proportional to their stiffness, hence the objective function. k2
Optimum k1 k1
kmax

27. Isolines
Isolines (level sets) connect points with equal function values:

28. The design space (cont.)
No feasible domain
Problem overconstrained:
no solution exists.
Dominated constraint
(redundant)
Just to introduce more terminology and concepts …

29. Design space (cont.) A and B inactive Interior optimum Objective
function
isolines A
A and B active
Objective
function
isolines
Optimum
B active, A inactive
Optimum
Objective
function
isolines B

30. Active constraint optimization
Idea of constraint activity at boundary optimum sometimes used in intuitive design optimization:
Fully stressed design (sizing / topology optimization)
Simultaneous failure mode theory
Careful: does not always give the optimal solution! F F

31. Problem characteristics
Study of objective and constraint functions:
simplify problem
discover incorrect problem formulation
choose suitable optimization algorithms
Properties:
Boundedness
Linearity
Convexity
Monotonicity

32. Boundedness
Proper bounds are necessary to avoid unrealistic solutions:
Example: aspirin pill design Objective: minimize dissolving time = maximize surface area (fixed volume) r h

33. Boundedness (cont.)
Volume equality constraint can be substituted, yielding: r f
Restrictions on r needed.
Note, surface minimization problem is well-defined!

34. Linearity
“A function f is linear if it satisfies f(x1+ x2) = f(x1)+ f(x2) and f(a x1) = a f(x1) for every two points x1, x2 in the domain, and all a”

35. Linearity (2)
Nonlinear objective functions can have multiple local optima: x1 x2 f x1 x2 f x
Challenge: finding the global optimum.

36. Problem characteristics
Study of objective and constraint functions:
simplify problem
discover incorrect problem formulation
choose suitable optimization algorithms
Properties:
Boundedness
Linearity
Convexity
Monotonicity

37. Boundedness Surface maximization of aspirin pill not well bounded: f r
Restrictions on r needed.
Note, surface minimization problem is well-defined!

38. Linearity
Nonlinear objective functions can have multiple local optima: x1 x2 f x1 x2 f x
Challenge: finding the global optimum.

39. Convexity
Convex function: any line connecting any 2 points on the graph lies above it (or on it):
Note difference between strictly convex and “normal” convex.
Linearity implies convexity (but not strict convexity)

40. Convexity (cont.) Convex set [Papalambros 4.27]:
“A set S is convex if for every two points x1, x2 in S, the connecting line also lies completely inside S”

41. Convexity (cont.)
Nonlinear constraint functions can result in nonconvex feasible domains: x2 x1
Nonconvex feasible domains can have multiple local boundary optima, even with linear objective functions!

42. Monotonicity Papalambros p. 99: f2 f1 x1 x2 Similar:
Function f is strictly monotonically increasing if: f(x2) > f(x1) for x2 > x1
weakly monotonically increasing if: f(x2)  f(x1) for x2 > x1
Similar for mon. decreasing f2 f1 x1 x2
Similar:
Note: monotonicity  convexity!
Linearity implies monotonicity

43. Optimization problem characteristics
Responses:
Boundedness
Linearity
Convexity
Monotonicity
Feasible domain:
Convexity

44. Example: tubular column designf g2 t

45. Optimization problem analysis
Motivation:
Simplification
Identify formulation errors early
Identify under- / overconstrained problems
Insight
Necessary conditions for existence of optimal solution
Basis: boundedness and constraint activity

46. Well-bounded functions – some definitions
Lower bound:
Greatest lower bound (glb): g f x
N = set of nonnegative numbers, P = set of positive and finite numbers
F(x*) = minimum, x* = minimizer
Minimum:
Minimizer: x*

47. Boundedness checking
Assumption: in engineering optimization problems, design variables are positive and finite
Define
Boundedness check:
Determine g+ for
Determine minimizers
Well bounded if
N: non-negative numbers
P: positive and finite numbers

48. Examples:
Bounded at zero
Asymptotically bounded

49. Air tank design Objective: minimize massh l
Not well bounded: constraints needed

50. Air tank constraints Minimum volume:
Min. head/radius ratio (ASME code):
Min. thickness/radius ratio (ASME code):
Room for nozzles: min. length
Space limitations: max. outside radius

51. Partial minimization & bounding constraints
Partial minimization: keep all variables constant but one. Example: air tank wall thickness t:
Conclusion:
f not well bounded from below
g3 bounds t from below

52. Constraint activity Removing constraint = relaxing problem
Solution set of relaxed problem without gi is Xi 1. 2. 3. A B
A and B active
Activity information can simplify problem:
Active: eliminate variable
Inactive: remove constraint

53. Constraint activity checking
Example: x2
f(1,x2,5) g3 g2
Partial minimization w.r.t. x1 shows that g1 bounds x1.
Idem x3 shows that g4 is inactive
Idem x2 shows that g = 1 (at x1=1, x3=5), X = 1, 3, 4, feasible 3, 4
Removing g3 is no problem. Removing g2 adds 1 to the solution. This means that g2 is semiactive.
Note, setting g2 active and using it to eliminate x2 (I.e. x2 = 2) yields the wrong solution!
Conclusion:
g1 active
g2 semiactive
g3 and g4 inactive

54. Activity and Monotonicity Theorem
“Constraint gi is active if and only if the minimum of the relaxed problem is lower than that of the original problem”
f(x) f
g(x) x g2 g1 x f g
f(x)
g(x)
“If f(x) and gi(x) all increase or decrease (weakly) w.r.t. x, the domain is not well constrained”

55. First Monotonicity Principle
“In a well-constrained minimization problem every variable that increases f is bounded below by at least one non-increasing active constraint” x f g
This principle can be used to find active constraints.
Exactly one bounding constraint: critical constraint
f(x)
g(x)

56. Air tank design Monotonicity analysis: What about l? Unclear.
Critical w.r.t. r
Critical w.r.t. h
Critical w.r.t. t
Both g1 and g4 could bound l from below. Further analysis is required.
What about l? Unclear.

57. Optimizing variables out
Critical constraints must be active:

58. Optimizing variables out
Critical constraints must be active:

59. Optimizing variables out
Critical constraints must be active:

60. Optimizing variables out
Critical constraints must be active:

61. Optimizing variables out
Critical constraints must be active:

62. Problem! Length not well bounded:
Additional constraint from above is needed:
Maximum plate width:

63. Air tank solution Length constraint is critical: must be active!
Result of Monotonicity Analysis:
Problem found, and fixed
Solution found without numerical optimization

64. Recognizing monotonicity
Some useful properties:
Sums:
Sums of similarly monotonic functions have the same monotonicity
Products:
Products of similarly monotonic functions have:
same monotonicity if
opposite monotonicity if

65. Recognizing monotonicity
More properties:
Powers:
Positive powers of monotonic functions have the same monotonicity, negative powers have opposite monotonicity
Composites:

66. Recognizing monotonicityf1
Integrals:
w.r.t. limits: a b x
w.r.t. integrand: f1 x y a b

67. Criticality Refined definitions:
# of variables critically bounded by constraint i
# of constraints possibly critically bounding variable j
Uncritical constraint 1
Uniquely critical constraint
>1
Multiple critical constraint
>1
Conditionally critical constraint

68. Air tank example
Multiple critical!
Critical w.r.t. r
Critical w.r.t. h
Critical w.r.t. t
Conditionally critical w.r.t. l
Emphasize that the contidionally critical constraints for l SEEM to indicate that l is properly bounded. However, as it turns out, it is not!
Multiple critical constraint can obscure boundedness! Eliminate if possible

69. Air tank example
Starting with eliminating r:

70. ? Air tank example New problem: Critical for h Critical for t
Conclusion: eliminate multiply critical constraints if possible, otherwise no conclusion w.r.t. proper boundedness can be made.

71. Air tank example Finally, after also eliminating h and t:
Not well bounded!
To be honest, I used Maple to do the work.
Conclusion: multiple critical constraint obscured ill-boundedness in l

72. Summary Optimization problem checking: Boundedness check of objective
Identify underconstrained problems
Monotonicity analysis
Identify not properly bounded problems
Identify critical constraints
Eliminate variables
Remove inactive constraints

73. But what about … Equality constraints: Example: Relaxed problem:
Active if all constraint variables in objective
Otherwise semi-active
Example: x1 x2 f 3 1
Relaxed problem:

74. More on nonobjective variables
Monotonicity Principle for nonobjective variables: “In a well-constrained minimization problem every nonobjective variable is bounded below by at least one non-increasing semiactive constraint and above by at least one non-decreasing semiactive constraint”
g(x) gi gj x

75. Nonobjective variables (2)
Other options:
Equality constraint
Single nonmonotonic constraint
h(x)
g(x) hi gi x x
See example in book (Papalambros p. 114)

76. Nonmonotonic functions
Monotonicity analysis difficult!
Sometimes regional monotonicity can be used
Concave constraints can split feasible domain:
g(x)
This gave a flavour of Monotonicity Analysis and what it can be used for. For special cases and more complicated situations, see book. gj gi x

77. Model preparation procedure (3.9)
Remove dominated constraints
Check boundedness for each design variable:
Objective monotonic? Constraints monotonic?
Critical constraints? Uniquely / conditionally / multiply?
If possible, eliminate active constraints, and repeat steps
Spending time on model checking usually pays off!