1. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 123 “True” Constrained Minimization

2. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 124 Classification Optimization algorithms for constrained optimization are often classified in primal and transformation methods: By a primal method of solution we mean a search method that works on the original problem directly by searching through the feasible region for the optimal solution. Transformation methods convert a constrained optimization problem to a sequence of unconstrained optimization problems. They include barrier and penalty function methods.

3. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 125 Sequential Linear Programming Sequential Linear Programming was developed in the early 1960s. Also known as Kelley's cutting plane method. Also known as Stewart and Griffith's Method of Approximate Programming. SLP is considered unattractive by theoreticians. However, the concept has proven to be quite powerful and efficient for engineering design. The basic concept is that we first linearize the objective and constraints and then solve this linear problem by an optimizer of choice.

4. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 126 SLP Algorithm Basic idea: Use linear approximations of the nonlinear functions and apply standard linear programming techniques. Process is repeated successively as the optimization process. Major concern: How far from the point of interest are these approximations valid? This problem is generally addressed by introducing move limits (often by trial and error). Note: Move limits depend on degree of nonlinearity. Traditional Stewart and Griffith Method uses first order Taylor expansion:

5. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 127 Variations on SLP Better results can be obtained by retaining the second-order terms of the Taylor series expansion, i.e.,

6. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 128 ALP Algorithm in DSIDES ALP algorithm is a variation on SLP. Some unique features: the use of second-order terms in linearization, the normalization of the constraints and goals and their transformation into generally well-behaved convex functions in the region of interest, an “intelligent” constraint suppression and accumulation scheme. ALP uses only diagonal terms of Hessian. Thus 2nd order Taylor series becomes:

7. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 129 Creation of a Hyperplane Hatched area represents the linear hyperplane in the three dimensional space. Note that g(x) is the objective function value.

8. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 130 Creation of a Second Improved Hyperplane Given the solution of the first approximation, one can make an improved hyperplane by using this solution and the existing quadratic approximation. e q = g h ( X F ) - g q ( X F ) Error test:

9. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 131 Constraint Accumulation Three dimensional view of constraint accumulation, i.e., you "remember" previous approximations of a constraint in order to improve the overall approximation.

10. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 132 Convexity Accumulation is only useful for convex constraints Why not for non-convex? Low degree of convexity

11. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 133 Adaptation of Convex Constraint New improved constraint approximation is added to first approximation of a constraint with high degree of convexity

12. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 134 Adaptation of Non-Convex Constraint Original (first) linear approximation is replaced (or modified) with new approximation.

13. Optimization in Engineering Design Georgia Institute of Technology Systems Realization Laboratory 135 Other ALP Features The software implementation has various features (see manual): Linear solver is a version of Revised Multiplex, hence lexicographic multi-objective optimization is possible. Generation of initial point and exploration of design space Adaptive reduced move (optional) Automatic constraint suppression (embodied in code) Perturbation step size control...