1. Computer Algorithms Integer Programming ECE 665 Professor Maciej Ciesielski By DFG

2. 2 Department of Electrical Engineering - UMASS Outline  Mathematical programming paradigm Linear Programming Integer Programming  Integer Programming Example Unimodularity LP -> IP Theorem Conclusion  Special Linear Programming with Integer Solutions Assignment Problem Network Flow Problem  Review  Conclusions

3. 3 Department of Electrical Engineering - UMASS Mathematical Programming Paradigm A mathematical program is an optimization problem of the form: Maximize (or Minimize) f(x) subject to: g(x) = 0 h(x) ≥ 0 where x = [x1,...xn] is a subset of R n, the functions g and h are called constraints, and f is called the objective function. Mathematical Programming paradigm

4. 4 Department of Electrical Engineering - UMASS Linear Programming (LP)  The goal of Linear programming is to: (Max)Minimize:C T x Subject to:Ax ≥ b x ≥ 0 Where C T is a coefficient vector for f, A is a constraint matrix, and b es a constraint vector.  The constraint set: {x | Ax ≥ b} is a convex polyhedron, and f is linear so it is convex. Therefore, LP has convexity, and the local min/max is the global min/max

5. 5 Department of Electrical Engineering - UMASS Integer Programming (IP)  The goal of Integer programming is to: (Max)Minimize:C T x Subject to:Ax  b x i integer  Typically, x i = 0,1  (0,1 Integer Programming)

6. 6 Department of Electrical Engineering - UMASS IP Example (Matching Problem) Given a graph G, find maximal set of edges in G, such that no two edges are adjacent to the same vertex.  Maximum matching: matching of maximum cardinality.  Weighted matching: matching with 13 2 e1e1 e2e2 e3e3

7. 7 Department of Electrical Engineering - UMASS IP Example (Matching Problem) 13 2 e1e1 e2e2 e3e3

8. 8 Department of Electrical Engineering - UMASS IP Example (Matching Problem) In matrix form: Where b = [ 1,…1]T, A = incidence matrix of G 1 2 3

9. 9 Department of Electrical Engineering - UMASS IP Example (Matching Problem) max 1x 1 + 1x 2 + 1x 3 One solution to this IP problem: x 1, x 2, x 3 = 0, 1 x 1 = 1, x 2 = 0, x 3 = 0 Other possible solutions: x 1 = 0, x 2 = 1, x 3 = 0, or x 1 = 0, x 2 = 0, x 3 = 1

10. 10 Department of Electrical Engineering - UMASS Question… Can the solution to this IP problem be obtained by dropping the integrality constraint: xi = 0, 1 And solving the LP problem instead? In our example, solution to the IP is not obtainable from LP. Reason: matrix A does not have certain property (total unimodularity) needed to guarantee integer solutions.

11. 11 Department of Electrical Engineering - UMASS If x i = 0, 1 is relaxed, such that x i  0, then the solution to the associated LP problem is non – integer: Reason: the A matrix Is not totally unimodular: |A| = - 2 Question…

12. 12 Department of Electrical Engineering - UMASS Given a constraint set in standard form where A, b are integer Partition A = [B/N ]; x = [x B, x N ] B is nonsingular m x m basis, N is non-basic Unimodularity: LP -> IP

13. 13 Department of Electrical Engineering - UMASS Basic solution is In particular, when B = I and B -1 = I then X B = b a solution can be obtained by inspection (as in the initial step of Simplex method). Unimodularity: LP -> IP

14. 14 Department of Electrical Engineering - UMASS Since x B = B –1 b with x N = 0, b integer A sufficient condition for a basic solution x B to be integer is that B –1 be an integer matrix Unimodularity: LP -> IP

15. 15 Department of Electrical Engineering - UMASS A square matrix B is called unimodular if D = |det B| = 1 An integer matrix A is totally unimodular if every square, nonsingular submatrix of A is unimodular. Equivalently: A is totally unimodular if every subdeterminant of A is 0, +1, or –1. Unimodularity

16. 16 Department of Electrical Engineering - UMASS Recall that for B nonsingular: Where B +, adjoint matrix = [ i j cofactor of element a i j in det A ] T B + and det B are integer if B –1 is integer Unimodularity

17. 17 Department of Electrical Engineering - UMASS Cofactor of a i j : Determinant obtained by omitting the i th row and the j th column of A and then multiplying by (-1) i + j. Unimodularity

18. 18 Department of Electrical Engineering - UMASS For B unimodular, B –1 integer: If A is totally unimodular, every basis matrix B is unimodular and every basic solution ( x B, y N ) = ( B –1 b, 0 ) Is integer. In particular, the optimal solution is integer Unimodularity

19. 19 Department of Electrical Engineering - UMASS If A is totally unimodular then every basic solution of Ax = b is integer. For LP’s with equality constraints total unimodularity is sufficient but not necessary. For LP with inequality constraints, Ax  b, total unimodularity of A is both necessary and sufficient for all extreme points of s = {x : Ax  b, x  0 } to be integer for every integer vector b. Theorem 1:

20. 20 Department of Electrical Engineering - UMASS Any IP with totally unimodular constraint matrix can be solved as an LP. Totally unimodular matrix : a i j = 0, +1, -1 Also, every determinant of A must be 0, +1 or -1 Conclusion:

21. 21 Department of Electrical Engineering - UMASS are LP with the property that they possess optimal solutions in integers. max flow min – cost flow assignment problem shortest path transportation problem Network flow problems: Special Linear Programs with Integer Solutions

22. 22 Department of Electrical Engineering - UMASS Assignment problem (special case of min – cost capacitated flow problems) [ m jobs x m men ] c i j = cost of assigning man i to job j Some Totally Unimodular Linear Programs

23. 23 Department of Electrical Engineering - UMASS s. to. Some Totally Unimodular Linear Programs

24. 24 Department of Electrical Engineering - UMASS X 11 X 12 X 13 X 21 X 22 X 23 X 31 X 32 X 33 Some Totally Unimodular Linear Programs

25. 25 Department of Electrical Engineering - UMASS In matrix notation: Ax = 1 where m2m m2m2... Exactly m 1’s in each row Exactly 2 1’s in each column Some Totally Unimodular Linear Programs

26. 26 Department of Electrical Engineering - UMASS a set of origins V 1 ; each origin i  V 1 ; supplies a 1 of commodity. a set of destinations V 2 ; each destination j  V 2 ; has a demand b i of commodity. cost per unit commodity; c ij associated with sending commodity through (i, j ). Given: Network Flow Problems

27. 27 Department of Electrical Engineering - UMASS Constraint Set: (Totally unimodular) Network Flow Problems

28. 28 Department of Electrical Engineering - UMASS Special case: for single source, single destination, max flow problem in any case:Ax  b Network Flow Problems

29. 29 Department of Electrical Engineering - UMASS No matter which problem it is: the general format is A’x  b It can be shown that If A is totally unimodular then [A/I ] is also totally unimodular The transpose of totally unimodular matrix is also totally unimodular. where A = incidence matrix of the corresponding digraph, totally unimodular. I = identity matrix, A’ is totally unimodular. Review

30. 30 Department of Electrical Engineering - UMASS Exploit a special structure of the problem (total unimodularity, etc.) to obtain integer solutions by solving simpler problems. Transform the problem to another problem for which an approximate solution is easier to find. Once the structure of the problem is well understood, use heuristic, but… stay away from brute force approach. No efficient algorithms exist for general Integer Programming problems Review

31. 31 Department of Electrical Engineering - UMASS Graph Theory Mathematical Optimization A large number of CAD problems can be cast in analytical form: For some problems – efficient algorithms exist. For others – need to resort to heuristic, suboptimal solutions. Some known successful heuristic approaches Simulated annealing LP rounding Conclusions