1. 1 Logic-Based Methods for Global Optimization J. N. Hooker Carnegie Mellon University, USA November 2003

2. 2 Basic Idea Assume the problem becomes convex when certain variables are fixed. If these variables are discrete, we can reformulate the problem as disjunctions of convex constraints. If some of them are continuous, discretize them to obtain an approximate global solution. Motivation is to take advantage of advanced solution methods: Branch-and-bound method chooses the appropriate disjunct in each constraint. Nonlinear programming method solves convex subproblems that result when disjuncts are chosen.

3. 3 Outline General form of problem Structural design example Disjunctive formulation Branch and bound with convex relaxations Big-M formulation Convex hull formulation Logic-based outer approximation Logic-based Benders decomposition Branch and bound with convex quasi-relaxations Realistic structural design problem Solution by MILP Solution by quasi-relaxation Other applications

4. 4 General Form of Problem If is continuous, discretize it, to get approximate global solution. Logical conditions on y Assume that when is fixed to, we get a convex problem convex functions of x Vector of functions

5. 5 Objective is defined in the constraints We assume one per constraint. Many problems have this form. If not, constraints can in principle be put into this form by change of variable.

6. 6 For example, consider Use the change of variable and the constraints have the desired form: One y j per constraint Logical condition

7. 7 Structural Design Example = compression of bar j displacement = = thickness (cross-sectional area) of bar j cost = cost of steel penalty for displacement load = 10 load = 20 Choose bar thickness that minimizes cost. This example is intended only to illustrate the algorithms. A more realistic model for structural design is presented at the end of the talk.

8. 8 Global optimization problem: Hooke’s law or many closely spaced values for continuous problem Can be written in desired form: displace- ment thickness

9. 9 How to Solve It? Can in principle use branch and bound by branching on. But continuous relaxations at nodes of the search tree are in general nonconvex. So we write the problem in disjunctive form.

10. 10 Disjunctive Formulation Now each disjunct is convex. We will solve by: Branch and bound with convex relaxations (use disjunctive programming or MINLP). Logic-based outer approximation with linear relaxations. Relaxations can be large when there are many disjunctions. In this case consider: Logic-based Benders decomposition with discrete relaxation. Branch and bound with convex quasi-relaxations (requires that constraint functions satisfy certain properties).

11. 11 Recall the example… Disjunctive formulation is: Disjuncts are convex (linear)

12. 12 Branch and Bound with Convex Relaxations Two convex relaxations of a disjunction Big-M : Write a big-M formulation with 0-1 variables and take its continuous relaxation (i.e., drop the integrality requirement on the 0-1 variables). Convex hull : Write a convex hull formulation with 0-1 variables and take its continuous relaxation. Two solution options Disjunctive programming : branch on disjunctions. Mixed integer nonlinear programming (MINLP): branch on 0-1 variables. Optimal value of relaxation provides a lower bound that is used in a branch-and-bound scheme.

13. 13 Big-M formulation of disjunction The disjunction: Big-M formulation: Where M v is a vector of valid upper bounds on the component functions of g(x,v). It is assumed that x is bounded above and below. To obtain relaxation, replace with

14. 14 Relaxation… Projection is Example of big-M relaxation Disjunction…

15. 15 The disjunction: Convex hull formulation of disjunction Stubbs & Mehrotra; Grossmann & Lee Assume each g(x,v) is bounded as well as convex. Also assume x L  x  x U Write every point in the relaxation as a convex combination of points satisfying the disjuncts Use change of variable Nonconvex

16. 16 Restore convexity by multiplying by This is a convex hull relaxation (i.e., projects onto convex hull in x-space). But disaggregation of x adds many new variables. To get 0-1 formulation, replace convex with

17. 17 Example of convex hull relaxation Disjunction… Convex hull relaxation…

18. 18 Solve structural design example with big-M formulation Big-M formulation: Disjunctive formulation: Solve by disjunctive programming or MINLP. Get optimal solution at the root node. Thus

19. 19 Solve structural design example with convex hull formulation Convex hull formulation: Disjunctive formulation: Solve by disjunctive programming or MINLP.

20. 20 Logic-Based Outer Approximation Türkay and Grossmann Allows one to use linear relaxations. But one must solve a mixed integer linear programming (MILP) master problem repeatedly. Solve a master problem containing 1 st -order approximations of the disjuncts to obtain a value for y. Solve with MILP, which uses linear relaxations. Solve the subproblem that results when y is fixed to this value, to get value for x. Compute 1 st -order approximations about previously obtained values of x, y. Continue until value of master problem  best value obtained in a subproblem so far. Begin with warm start by precomputing 1 st -order approximations about several values of (x,y).

21. 21 Disjunctive formulation again: The master problem in iteration K + 1 is where (x k,y k ) are solutions from previous iterations. The nonlinear subproblem in iteration K is

22. 22 Solve structural design example with logic-based outer approximation Master problem: Disjunctive formulation again: Disjuncts already linear

23. 23 Solve master problem as MILP (Big-M formulation):` For warm start, solve subproblem for 2 y’s: y 1 = (1,1), which yields x 1 = (20,20) y 2 = (2,2), which yields x 2 = (5,10). This results in the master problem:

24. 24 Solve master problem and get which implies Subproblem solution is Next master problem is Solution is and the algorithm terminates with y = (1,2). same new

25. 25 Logic-Based Benders Decomposition Hooker and Ottosson Can be useful when variables have a large number of discrete values, resulting in a large number of disjuncts. Convergence can be slow. Solve a master problem for y. The master problem incompletely describes the projection of the original problem onto the y-space. Solve the subproblem that results when y is fixed to this value. Obtain Benders cut from inference dual of the subproblem. Add the cut to the master problem to rule out some solutions that are no better than the previous one. Continue until the master and subproblem converge in value. Best to have a warm start with “don’t-be-stupid” constraints involving y.

26. 26 Disjunctive formulation again: The nonlinear subproblem in iteration K is Lagrange multiplier The master problem in iteration K + 1 is Optimal value = Logical Benders cuts

27. 27 Solve structural design example with logic-based Benders decomposition Initial master problem: One solution is Solve subproblem: Corresponds to y 1 = 1 Corresponds to y 2 = 1 Lagrange multipliers Subproblem solution is Don’t-be-stupid constraint

28. 28 Since the master problem is Solution is Continue in this fashion. Master problem in iteration 4 is: Solution is The algorithm terminates with y = (1,2). same

29. 29 Branch and Bound with Convex Quasi-Relaxations Does not require disjunctive formulation and is therefore useful when there are many discrete values. But the constraint functions must have a certain form. Solve the problem by branch and bound. Obtain bounds from quasi-relaxations at each node. Given problem P: a problem Q: is a quasi-relaxation of P if for any feasible solution x of P, there is a feasible solution x of Q with f (x )  f (x). Thus one can obtain a valid lower bound by solving a quasi-relaxation.

30. 30 Consider the problem, Theorem. Suppose that each is either (a) convex [for (i,j)  J 1 ] or (b) concave in y j and homogeneous in x: Then the following is a convex quasi-relaxation : [for (i,j)  J 2 ]. Suppose also that

31. 31 Why? Take any feasible solution of To obtain a feasible solution of do the following: Then concavity homogeneity

32. 32 So we have a feasible solution of the quasi- relaxation with value that is less than or equal to (in fact equal to) that of the original problem. convex, because g j (x,y) is convex convex, because g j (x,y) is convex in x satisfied, by above argument satisfied, by construction

33. 33 Solve continuous version of structural design example with quasi-relaxations Original formulation: Put in proper form: convex Concave in y j & homogeneous in 1 st argument (s j, x j ) Discretize

34. 34 The quasi-relaxation is: Can now re-aggregate s j :

35. 35 Beginning of branch-and-bound tree Root node x 0 = 1177.8  = (0.667,0.667) y = (1,1) x 0 = 1322  = (0,0.667) y = (1,1) x 0 =1900  = (0,0) y = (1,1) feasible solution y 1  [0,1] y 1  [1.1,3] x 0 = 1283  = (0.816,0.667) y = (1.45,1) y 2  [0,1] y 2  [1.1,3] x 0 =1352  = (0,0.816) y = (1,1.45) Total 63 nodes out of 31  31 possible solutions. Get y = (1.1, 2.0) with z 0 = 1394.5 Global optimum is y = (1.126, 1.972) with z 0 = 1394.1

36. 36 Degree of freedom i Load Bar j Length Elongation Cross- sectional area Displacement Realistic Structural Design Problem Hooke’s law Equilibrium Compatibility Elongation bounds Displacement bounds

37. 37 Solution as MI(N)LP Ghattas and Grossmann The disjunctive formulation is Since everything is linear, the big-M and convex hull formulations are linear. Can solve as a mixed integer linear programming (MILP) problem. Discrete sizes for bar j

38. 38 The convex hull MILP model is

39. 39 Solution with convex quasi-relaxations Bollapragada, Ghattas and Hooker Check that the problem has the right form: convex Concave (linear) in y j and homogeneous in v j, f j

40. 40 The quasi-relaxation is

41. 41 10-bar cantilever truss 25-bar electrical transmission tower Some problem instances

42. 42 72-bar building Use symmetries to help solve problem

43. 43 Computational Results (seconds) Problem InstanceMILP (CPLEX) Quasi- relaxation 10-bar cantilever truss 11 discrete areas 1 load1.30.3 1 load, wider stress bounds1.60.3 1 load, wider stress bounds2.61.2 1 load, wider stress bounds2.61.4 2 loads23.65.8 1 load + displacement bounds1089.467.5 2 loads + displacement bounds13743.91654.0 25-bar transmission tower 2 loads, 11 discrete areas271.7225.8 Building 11 discrete areas 2 loads 72 bars12692.7207.9 90 bars*168.9 108 bars*329.4 *No solution after 20 hours (72,000 seconds)

44. 44 Applications of Logic-Based Methods for Nonlinear Global Optimization Logic-based outer approximation applied to chemical processing network design Quesada and Grossmann 1992; Türkay and Grossmann 1996 Disjunctive programming applied to chemical processing network design Vecchietti and Grossmann 1999 Convex quasi-relaxations applied to truss structure design Bollapragada, Ghattas and Hooker 2001 Disjunctive programming applied to tray placement in distillation columns Barttfeld, Aguire and Grossmann 2003 Logic-based Benders decomposition applied to planning and scheduling (linear case) Hooker 2000, 2003.; Jain and Grossmann 2001

45. 45 Surveys I. E. Grossmann, Review of nonlinear mixed- integer and disjunctive programming techniques for process systems engineering, Carnegie Mellon University, June 2001 J. N. Hooker, Logic-Based Methods for Optimization, John Wiley & Sons, 2000.