1. Spring 07, Mar 13, 15 ELEC 7770: Advanced VLSI Design (Agrawal) 1 ELEC 7770 Advanced VLSI Design Spring 2007 Linear Programming – A Mathematical Optimization Technique Vishwani D. Agrawal James J. Danaher Professor ECE Department, Auburn University Auburn, AL 36849 vagrawal@eng.auburn.edu http://www.eng.auburn.edu/~vagrawal/COURSE/E7770_Spr07

2. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)2 What is Linear Programming  Linear programming (LP) is a mathematical method for selecting the best solution from the available solutions of a problem.  Method:  State the problem and define variables whose values will be determined.  Develop a linear programming model:  Write the problem as an optimization formula (a linear expression to be minimized or maximized)  Write a set of linear constraints  An available LP solver (computer program) gives the values of variables.

3. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)3 Types of LPs  LP – all variables are real.  ILP – all variables are integers.  MILP – some variables are integers, others are real.  A reference:  S. I. Gass, An Illustrated Guide to Linear Programming, New York: Dover, 1990.

4. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)4 A Single-Variable Problem  Consider variable x  Problem: find the maximum value of x subject to constraint, 0 ≤ x ≤ 15.  Solution: x = 15.

5. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)5 Single Variable Problem (Cont.)  Consider more complex constraints:  Maximize x, subject to following constraints  x ≥ 0(1)  5x ≤ 75(2)  6x ≤ 30(3)  x ≤ 10(4) 051015x (1) (2) (3) (4) All constraints satisfied Solution, x = 5

6. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)6 A Two-Variable Problem  Manufacture of x 1 chairs and x 2 tables:  Maximize profit, P = 45x 1 + 80x 2 dollars  Subject to resource constraints:  400 boards of wood,5x 1 + 20x 2 ≤ 400(1)  450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2)  x 1 ≥ 0(3)  x 2 ≥ 0 (4)

7. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)7 Solution: Two-Variable Problem Chairs, x 1 Tables, x 2 (1) (2) 0 10 20 30 40 50 60 70 80 90 40 30 20 10 0 (24, 14) Profit increasing decresing P = 2200 P = 0 Best solution: 24 chairs, 14 tables Profit = 45×24 + 80×14 = 2200 dollars

8. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)8 Change Chair Profit, $64/Unit  Manufacture of x 1 chairs and x 2 tables:  Maximize profit, P = 64x 1 + 80x 2 dollars  Subject to resource constraints:  400 boards of wood,5x 1 + 20x 2 ≤ 400(1)  450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2)  x 1 ≥ 0(3)  x 2 ≥ 0 (4)

9. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)9 Solution: $64 Profit/Chair Chairs, x 1 Tables, x 2 (1) (2) Profit increasing decresing P = 2880 P = 0 Best solution: 45 chairs, 0 tables Profit = 64×45 + 80×0 = 2880 dollars 0 10 20 30 40 50 60 70 80 90 (24, 14) 40 30 20 10 0

10. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)10 Primal-Dual Problems  Primal problem  Fixed resources  Maximize profit  Variables:  x 1 (number of chairs)  x 2 (number of tables)  Maximize profit 45x 1 + 80x 2  Subject to:  5x 1 + 20x 2 ≤ 400  10x 1 + 15x 2 ≤ 450  x 1 ≥ 0  x 2 ≥ 0  Solution:  x 1 = 24 chairs, x 2 = 14 tables  Profit = $2200  Dual Problem  Fixed profit  Minimize cost  Variables:  w 1 ($ cost/board of wood)  w 2 ($ cost/man-hour)  Minimize cost 400w1 + 450w2  Subject to:  5w 1 +10w 2 ≥ 45  20w 1 + 15w 2 ≥ 80  w 1 ≥ 0  w 2 ≥ 0  Solution:  w 1 = $1, w 2 = $4  Cost = $2200

11. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)11 The Duality Theorem  If the primal has a finite optimal solution, so does the dual, and the optimum values of the objective functions are equal.

12. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)12 LP for n Variables n minimize Σ cj xjObjective function j =1 n subject to Σ aij xj ≤ bi, i = 1, 2,..., m j =1 n Σ cij xj = di, i = 1, 2,..., p j =1 Variables: xj Constants: cj, aij, bi, cij, di

13. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)13 Algorithms for Solving LP  Simplex method  G. B. Dantzig, Linear Programming and Extension, Princeton, New Jersey, Princeton University Press, 1963.  Ellipsoid method  L. G. Khachiyan, “A Polynomial Algorithm for Linear Programming,” Soviet Math. Dokl., vol. 20, pp. 191-194, 1984.  Interior-point method  N. K. Karmarkar, “A New Polynomial-Time Algorithm for Linear Programming,” Combinatorica, vol. 4, pp. 373-395, 1984.  Course website of Prof. Lieven Vandenberghe (UCLA), http://www.ee.ucla.edu/ee236a/ee236a.html http://www.ee.ucla.edu/ee236a/ee236a.html

14. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)14 Basic Ideas of Solution methods Constraints Extreme points Objective function Constraints Extreme points Objective function Simplex: search on extreme points. Interior-point methods: Successively iterate with interior spaces of analytic convex boundaries.

15. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)15 Integer Linear Programming (ILP)  Variables are integers.  Complexity is exponential – higher than LP.  LP relaxation  Convert all variables to real, preserve ranges.  LP solution provides guidance.  Rounding LP solution can provide a non-optimal solution.

16. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)16 Solving TSP: Five Cities Distances (dij) in miles (symmetric TSP, general TSP is asymmetric) City j=1 j=1 j=2 j=2j=3j=4j=5 i=1 i=1018101227 i=2 i=218051220 i=3 i=310501519 i=4 i=412121506 i=5 i=527201960

17. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)17 Search Space: No. of Tours  Asymmetric TSP tours  Five-city problem: 4 × 3 × 2 × 1 = 24 tours  Nine-city problem: 362,880 tours  14-city problem: 87,178,291,200 tours

18. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)18 A Greedy Heuristic Solution City j=1 j=1 j=2 j=2j=3j=4j=5 i=1 i=1(start)018101227 i=2 i=218051220 i=3 i=310501519 i=4 i=412121506 i=5 i=527201960 Tour length = 10 + 5 + 12 + 6 + 27 = 60 miles (non-optimal)

19. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)19 ILP Variables, Constants and Constraints 1 3 2 5 4 d14 = 12 d15 = 27 d12 = 18 d13 = 10 x14 ε [0,1] x15 ε [0,1] x12 ε [0,1] x13 ε [0,1] x12 + x13 + x14 + x15 = 2 four other similar equations

20. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)20 Objective Function and ILP Solution 5 i - 1 Minimize ∑ ∑ xij × dij i = 1 j = 1 xij xij j=1 j=12345 i=1 i=100100 210000 301000 400001 500010 ∑ xij = 2 for all i j ≠ i

21. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)21 ILP Solution 1 3 2 5 4 d13 = 10 d45 = 6 Total length = 45 but not a single tour d54 = 6 d21 = 18 d32 = 5

22. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)22 Additional Constraints for a Single Tour  Following constraints prevent subtours. For any subset S of cities, the tour must enter and exit that subset: ∑ xij ≥ 2 for all S, |S| < 5 i ε S j ε S

23. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)23 ILP Solution 1 3 2 5 4 d13 = 10 d41 = 12 Total length = 53 d54 = 6 d25 = 20 d32 = 5

24. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)24 ILP Example: Test Minimization  A combinational circuit has n test vectors that detect m faults. Each test detects a subset of faults. Find the smallest subset of test vectors that detects all m faults.  ILP model:  Assign an integer variable ti ε [0,1] to ith test vector such that ti = 1, if we select ti, otherwise ti= 0.  Define an integer constant fij ε [0,1] such that fij = 1, if ith vector detects jth fault, otherwise fij = 0. Values of constants fij are determined by fault simulation.

25. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)25 Test Minimization by ILP n minimize Σ ti Objective function i=1 n subject to Σ fij ti ≥ 1, j = 1, 2,..., m i=1

26. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)26 3V3F: A 3-Vector 3-Fault Example fiji=1i=2i=3 j=1110 j=2011 j=3101 Test vector i Fault j ε Variables: t1, t2, t3 ε [0,1] Minimize t1 + t2 + t3 Subject to: t1 + t2 ≥ 1 t2 + t3 ≥ 1 t1 + t3 ≥ 1

27. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)27 3V3F: Solution Space Non-optimum solution t1 t2 t3 1 1 1 LP solution (0.5, 0.5, 0.5) and recursive rounding ILP solutions (optimum)

28. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)28 Characteristics of ILP  Worst-case complexity is exponential in number of variables.  Linear programming (LP) relaxation, where integer variables are treated as real, gives a lower bound on the objective function.  Recursive rounding of relaxed LP solution to nearest integers gives an approximate solution to the ILP problem.  K. R. Kantipudi and V. D. Agrawal, “A Reduced Complexity Algorithm for Minimizing N-Detect Tests,” Proc. 20 th International Conf. VLSI Design, January 2007, pp. 492-497.

29. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)29 3V3F: LP Relaxation and Rounding ε ILP – Variables: t1, t2, t3 ε [0,1] Minimize t1 + t2 + t3 Subject to: t1 + t2 ≥ 1 t2 + t3 ≥ 1 t1 + t3 ≥ 1 ε LP relaxation: t1, t2, t3 ε (0.0, 1.0) Solution: t1 = t2 = t3 = 0.5 Recursive rounding: (1) round one variable, t1 = 1.0 Two-variable LP problem: Minimize t2 + t3 subject to t2 + t3 ≥ 1.0 LP solution t2 = t3 = 0.5 (2) round a variable, t2 = 1.0 ILP constraints are satisfied solution is t1 = 1, t2 = 1, t3 = 0

30. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)30 Recursive Rounding Algorithm 1.Obtain a relaxed LP solution. Stop if each variable in the solution is an integer. 2.Round the variable closest to an integer. 3.Remove any constraints that are now unconditionally satisfied. 4.Go to step 1.

31. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)31 Recursive Rounding  ILP has exponential complexity.  Recursive rounding:  ILP is transformed into k LPs with progressively reducing number of variables.  Number of LPs, k, is the size of the final solution, i.e., the number of non-zero variables in the test minimization problem.  Recursive rounding complexity is k × O(n p ), where k ≤ n, n is number of variables.

32. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)32 Four-Bit ALU Circuit Initial vectors ILP Recursive rounding Vectors CPU s Vectors 285140.65140.42 400131.07131.00 500124.38133.00 1,000124.17123.00 5,0001212.95129.00 10,0001234.611217.0 16,3841287.471237.0

33. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)33 CPU Time: ILP vs. Recursive Rounding 0 5,000 10,000 15,000 Vectors 100 75 50 25 0 ILP Recursive Rounding CPU s

34. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)34 N-Detect Tests (N = 5) Circuit Unoptimized vectors Relaxed LP/Recur. rounding ILP (exact) Lower bound Min. vectors CPU s Min. vectors CPU s c432 c432608196.381971.01971.0 c499 c499379260.002601.22602.3 c880 c8801,023125.9712814.0127881.8 c1355 c1355755420.004203.24204.4 c1908 c19081,055543.005434.65436.9 c2670 c2670959477.004774.74777.2 c3540 c35401,971467.2547772.047120008.5 c5315 c53151,079374.3337718.037640.7 c6288 c628824352.525739.05734740.0 c7552 c75522,165841.0084152.0841114.3

35. Spring 07, Mar 13, 15ELEC 7770: Advanced VLSI Design (Agrawal)35 Finding LP/ILP Solvers  R. Fourer, D. M. Gay and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, South San Francisco, California: Scientific Press, 1993. Several of programs described in this book are available to Auburn users.  MATLAB?  Search the web. Many programs with small number of variables can be downloaded free.