1. Copyright Agrawal, 2007 ELEC6270 Fall 07, Lecture 8 1 ELEC 5270/6270 Fall 2007 Low-Power Design of Electronic Circuits Linear Programming – A Mathematical Optimization Technique Vishwani D. Agrawal James J. Danaher Professor Dept. of Electrical and Computer Engineering Auburn University, Auburn, AL 36849 vagrawal@eng.auburn.edu http://www.eng.auburn.edu/~vagrawal/COURSE/E6270_Fall07/course.html

2. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 82 What is Linear Programming Linear programming (LP) is a mathematical method for selecting the best solution from the available solutions of a problem. Linear programming (LP) is a mathematical method for selecting the best solution from the available solutions of a problem. Method: Method: State the problem and define variables whose values will be determined. State the problem and define variables whose values will be determined. Develop a linear programming model: Develop a linear programming model: Write the problem as an optimization formula (a linear expression to be minimized or maximized) Write the problem as an optimization formula (a linear expression to be minimized or maximized) Write a set of linear constraints Write a set of linear constraints An available LP solver (computer program) gives the values of variables. An available LP solver (computer program) gives the values of variables.

3. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 83 Types of LPs LP – all variables are real. LP – all variables are real. ILP – all variables are integers. ILP – all variables are integers. MILP – some variables are integers, others are real. MILP – some variables are integers, others are real. A reference: A reference: S. I. Gass, An Illustrated Guide to Linear Programming, New York: Dover, 1990. S. I. Gass, An Illustrated Guide to Linear Programming, New York: Dover, 1990.

4. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 84 A Single-Variable Problem Consider variable x Consider variable x Problem: find the maximum value of x subject to constraint, 0 ≤ x ≤ 15. Problem: find the maximum value of x subject to constraint, 0 ≤ x ≤ 15. Solution: x = 15. Solution: x = 15. 0 15 Constraint satisfied x Solution x = 15

5. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 85 Single Variable Problem (Cont.) Consider more complex constraints: Consider more complex constraints: Maximize x, subject to following constraints: Maximize x, subject to following constraints: x ≥ 0(1) x ≥ 0(1) 5x ≤ 75(2) 5x ≤ 75(2) 6x ≤ 30(3) 6x ≤ 30(3) x ≤ 10(4) x ≤ 10(4) 051015x (1) (2) (3) (4) All constraints satisfied Solution, x = 5

6. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 86 A Two-Variable Problem Manufacture of chairs and tables: Manufacture of chairs and tables: Resources available: Resources available: Material: 400 boards of wood Material: 400 boards of wood Labor: 450 man-hours Labor: 450 man-hours Profit: Profit: Chair: $45 Chair: $45 Table: $80 Table: $80 Resources needed: Resources needed: Chair Chair 5 boards of wood 5 boards of wood 10 man-hours 10 man-hours Table Table 20 boards of wood 20 boards of wood 15 man-hours 15 man-hours Problem: How many chairs and how many tables should be manufactured to maximize the total profit? Problem: How many chairs and how many tables should be manufactured to maximize the total profit?

7. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 87 Formulating Two-Variable Problem Manufacture x 1 chairs and x 2 tables to maximize profit: Manufacture x 1 chairs and x 2 tables to maximize profit: P = 45x 1 + 80x 2 dollars Subject to given resource constraints: Subject to given resource constraints: 400 boards of wood,5x 1 + 20x 2 ≤ 400(1) 400 boards of wood,5x 1 + 20x 2 ≤ 400(1) 450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2) 450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2) x 1 ≥ 0(3) x 1 ≥ 0(3) x 2 ≥ 0 (4) x 2 ≥ 0 (4)

8. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 88 Solution: Two-Variable Problem Chairs, x 1 Tables, x 2 (1) (2) 0 10 20 30 40 50 60 70 80 90 40 30 20 10 0 (24, 14) Profit increasing decresing P = 2200 P = 0 Best solution: 24 chairs, 14 tables Profit = 45×24 + 80×14 = 2200 dollars (3) (4) Material constraint Man-power constraint

9. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 89 Change Profit of Chair to $64/Unit Manufacture x 1 chairs and x 2 tables to maximize profit: Manufacture x 1 chairs and x 2 tables to maximize profit: P = 64x 1 + 80x 2 dollars Subject to given resource constraints: Subject to given resource constraints: 400 boards of wood,5x 1 + 20x 2 ≤ 400(1) 400 boards of wood,5x 1 + 20x 2 ≤ 400(1) 450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2) 450 man-hours of labor,10x 1 + 15x 2 ≤ 450(2) x 1 ≥ 0(3) x 1 ≥ 0(3) x 2 ≥ 0 (4) x 2 ≥ 0 (4)

10. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 810 Solution: $64 Profit/Chair Chairs, x 1 Tables, x 2 (1) (2) Profit increasing decresing P = 2880 P = 0 Best solution: 45 chairs, 0 tables Profit = 64×45 + 80×0 = 2880 dollars 0 10 20 30 40 50 60 70 80 90 (24, 14) 40 30 20 10 0 (3) (4) Material constraint Man-power constraint

11. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 811 Primal-Dual Problems Primal problem Primal problem Fixed resources Fixed resources Maximize profit Maximize profit Variables: Variables: x 1 (number of chairs) x 1 (number of chairs) x 2 (number of tables) x 2 (number of tables) Maximize profit 45x 1 +80x 2 Maximize profit 45x 1 +80x 2 Subject to: Subject to: 5x 1 + 20x 2 ≤ 400 5x 1 + 20x 2 ≤ 400 10x 1 + 15x 2 ≤ 450 10x 1 + 15x 2 ≤ 450 x 1 ≥ 0 x 1 ≥ 0 x 2 ≥ 0 x 2 ≥ 0 Solution: Solution: x 1 = 24 chairs, x 2 = 14 tables x 1 = 24 chairs, x 2 = 14 tables Profit = $2200 Profit = $2200 Dual Problem Dual Problem Fixed profit Minimize value Variables: Variables: w 1 ($ value/board of wood) w 2 ($ value/man-hour) Minimize value 400w 1 +450w 2 Minimize value 400w 1 +450w 2 Subject to: Subject to: 5w 1 + 10w 2 ≥ 45 20w 1 + 15w 2 ≥ 80 w 1 ≥ 0 w 2 ≥ 0 Solution: Solution: w 1 = $1, w 2 = $4 value = $2200

12. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 812 The Duality Theorem If the primal has a finite optimal solution, so does the dual, and the optimum values of the objective functions are equal. If the primal has a finite optimal solution, so does the dual, and the optimum values of the objective functions are equal.

13. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 813 LP for n Variables n minimize Σ cj xjObjective function j =1 n subject to Σ aij xj ≤ bi, i = 1, 2,..., m j =1 n Σ cij xj = di, i = 1, 2,..., p j =1 Variables: xj Constants: cj, aij, bi, cij, di

14. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 814 Algorithms for Solving LP Simplex method Simplex method G. B. Dantzig, Linear Programming and Extension, Princeton, New Jersey, Princeton University Press, 1963. G. B. Dantzig, Linear Programming and Extension, Princeton, New Jersey, Princeton University Press, 1963. Ellipsoid method Ellipsoid method L. G. Khachiyan, “A Polynomial Algorithm for Linear Programming,” Soviet Math. Dokl., vol. 20, pp. 191-194, 1984. L. G. Khachiyan, “A Polynomial Algorithm for Linear Programming,” Soviet Math. Dokl., vol. 20, pp. 191-194, 1984. Interior-point method Interior-point method N. K. Karmarkar, “A New Polynomial-Time Algorithm for Linear Programming,” Combinatorica, vol. 4, pp. 373-395, 1984. N. K. Karmarkar, “A New Polynomial-Time Algorithm for Linear Programming,” Combinatorica, vol. 4, pp. 373-395, 1984. Course website of Prof. Lieven Vandenberghe (UCLA), http://www.ee.ucla.edu/ee236a/ee236a.html Course website of Prof. Lieven Vandenberghe (UCLA), http://www.ee.ucla.edu/ee236a/ee236a.html http://www.ee.ucla.edu/ee236a/ee236a.html

15. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 815 Basic Ideas of Solution methods Constraints Extreme points Objective function Constraints Extreme points Objective function Simplex: search on extreme points. Interior-point methods: Successively iterate with interior spaces of analytic convex boundaries.

16. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 816 Integer Linear Programming (ILP) Variables are integers. Variables are integers. Complexity is exponential – higher than LP. Complexity is exponential – higher than LP. LP relaxation LP relaxation Convert all variables to real, preserve ranges. Convert all variables to real, preserve ranges. LP solution provides guidance. LP solution provides guidance. Rounding LP solution can provide a non- optimal solution. Rounding LP solution can provide a non- optimal solution.

17. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 817 Solving TSP: Five Cities Distances (dij) in miles (symmetric TSP, general TSP is asymmetric) City j=1 j=1 j=2 j=2j=3j=4j=5 i=1 i=1018101227 i=2 i=218051220 i=3 i=310501519 i=4 i=412121506 i=5 i=527201960

18. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 818 Search Space: No. of Tours Asymmetric TSP tours Asymmetric TSP tours Five-city problem: 4 × 3 × 2 × 1 = 24 tours Five-city problem: 4 × 3 × 2 × 1 = 24 tours Nine-city problem: 362,880 tours Nine-city problem: 362,880 tours 14-city problem: 87,178,291,200 tours 14-city problem: 87,178,291,200 tours 50-city problem: 49! = 6.08×10 tours 50-city problem: 49! = 6.08×10 62 tours Time for enumerative search assuming 1 μs per tour evaluation=1.93×10 years Time for enumerative search assuming 1 μs per tour evaluation=1.93×10 55 years

19. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 819 A Greedy Heuristic Solution City j = 1 j = 1 j = 2 j = 2 j = 3 j = 4 j = 5 i = 1 i = 1(start)018101227 i = 2 i = 218051220 i = 3 i = 310501519 i = 4 i = 412121506 i = 5 i = 527201960 Tour length = 10 + 5 + 12 + 6 + 27 = 60 miles (non-optimal)

20. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 820 ILP Variables, Constants and Constraints 1 3 2 5 4 d14 = 12 d15 = 27 d12 = 18 d13 = 10 x14 ε [0,1] x15 ε [0,1] x12 ε [0,1] x13 ε [0,1] x12 + x13 + x14 + x15 = 2 four other similar equations Integer variables: xij = 1, travel i to j xij = 0, do not travel i to j Real variables: dij = distance from i to j

21. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 821 Objective Function and ILP Solution 5 i - 1 Minimize ∑ ∑ xij × dij i = 1 j = 1 xij xij j=1 j=12345 i=1 i=100100 210000 301000 400001 500010 ∑ xij = 2 for all i j ≠ i

22. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 822 ILP Solution 1 3 2 5 4 d13 = 10 d45 = 6 Total length = 45 but not a single tour d54 = 6 d21 = 18 d32 = 5

23. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 823 Additional Constraints for Single Tour Following constraints prevent split tours. For any subset S of cities, the tour must enter and exit that subset: Following constraints prevent split tours. For any subset S of cities, the tour must enter and exit that subset: ∑ xij ≥ 2 for all S, |S| < 5 i ε S j ε S Subset Remaining set At least two arrows must cross this boundary.

24. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 824 ILP Solution 1 3 2 5 4 d13 = 10 d41 = 12 Total length = 53 d54 = 6 d25 = 20 d32 = 5

25. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 825 ILP Example: Test Minimization A combinational circuit has n test vectors that detect m faults. Each test detects a subset of faults. Find the smallest subset of test vectors that detects all m faults. A combinational circuit has n test vectors that detect m faults. Each test detects a subset of faults. Find the smallest subset of test vectors that detects all m faults. ILP model: ILP model: Assign an integer variable ti ε [0,1] to ith test vector such that ti = 1, if we select ti, otherwise ti= 0. Assign an integer variable ti ε [0,1] to ith test vector such that ti = 1, if we select ti, otherwise ti= 0. Define an integer constant fij ε [0,1] such that fij = 1, if ith vector detects jth fault, otherwise fij = 0. Values of constants fij are determined by fault simulation. Define an integer constant fij ε [0,1] such that fij = 1, if ith vector detects jth fault, otherwise fij = 0. Values of constants fij are determined by fault simulation.

26. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 826 Test Minimization by ILP n minimize Σ ti Objective function i=1 n subject to Σ fij ti ≥ 1, j = 1, 2,..., m i=1

27. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 827 3V3F: A 3-Vector 3-Fault Example fiji=1i=2i=3 j=1110 j=2011 j=3101 Test vector i Fault j ε Variables: t1, t2, t3 ε [0,1] Minimize t1 + t2 + t3 Subject to: t1 + t2 ≥ 1 t2 + t3 ≥ 1 t1 + t3 ≥ 1

28. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 828 3V3F: Solution Space Non-optimum solution t1 t2 t3 1 1 1 1 st LP solution (0.5, 0.5, 0.5) ILP solutions (optimum) Rounding and 2 nd ILP solution (1.0, 0.5, 0.5) Rounding and 3 rd LP solution (1.0, 1.0, 0.0)

29. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 829 Characteristics of ILP Worst-case complexity is exponential in number of variables. Worst-case complexity is exponential in number of variables. Linear programming (LP) relaxation, where integer variables are treated as real, gives a lower bound on the objective function. Linear programming (LP) relaxation, where integer variables are treated as real, gives a lower bound on the objective function. Recursive rounding of relaxed LP solution to nearest integers gives an approximate solution to the ILP problem. Recursive rounding of relaxed LP solution to nearest integers gives an approximate solution to the ILP problem. K. R. Kantipudi and V. D. Agrawal, “A Reduced Complexity Algorithm for Minimizing N-Detect Tests,” Proc. 20 th International Conf. VLSI Design, January 2007, pp. 492-497. K. R. Kantipudi and V. D. Agrawal, “A Reduced Complexity Algorithm for Minimizing N-Detect Tests,” Proc. 20 th International Conf. VLSI Design, January 2007, pp. 492-497.

30. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 830 3V3F: LP Relaxation and Rounding ε ILP – Variables: t1, t2, t3 ε [0,1] Minimize t1 + t2 + t3 Subject to: t1 + t2 ≥ 1 t2 + t3 ≥ 1 t1 + t3 ≥ 1 ε LP relaxation: t1, t2, t3 ε (0.0, 1.0) Solution: t1 = t2 = t3 = 0.5 Recursive rounding: (1) round one variable, t1 = 1.0 Two-variable LP problem: Minimize t2 + t3 subject to t2 + t3 ≥ 1.0 LP solution t2 = t3 = 0.5 (2) round a variable, t2 = 1.0 ILP constraints are satisfied solution is t1 = 1, t2 = 1, t3 = 0

31. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 831 Recursive Rounding Algorithm 1.Obtain a relaxed LP solution. if each variable in the solution is an integer. 1.Obtain a relaxed LP solution. Stop if each variable in the solution is an integer. 2.Round the variable closest to an integer. 3.Remove any constraints that are now unconditionally satisfied. 4.Go to step 1.

32. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 832 Recursive Rounding ILP has exponential complexity. ILP has exponential complexity. Recursive rounding: Recursive rounding: ILP is transformed into k LPs with progressively reducing number of variables. ILP is transformed into k LPs with progressively reducing number of variables. Number of LPs, k, is the size of the final solution, i.e., the number of non-zero variables in the test minimization problem. Number of LPs, k, is the size of the final solution, i.e., the number of non-zero variables in the test minimization problem. Recursive rounding complexity is k × O(n p ), where k ≤ n, n is number of variables. Recursive rounding complexity is k × O(n p ), where k ≤ n, n is number of variables.

33. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 833 Four-Bit ALU Circuit Initial vectors ILP Recursive rounding Vectors CPU s Vectors 285140.65140.42 400131.07131.00 500124.38133.00 1,000124.17123.00 5,0001212.95129.00 10,0001234.611217.0 16,3841287.471237.0

34. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 834 ILP vs. Recursive Rounding 0 5,000 10,000 15,000 Vectors 100 75 50 25 0 ILP Recursive Rounding CPU s

35. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 835 N-Detect Tests (N = 5) Circuit Unoptimized vectors Relaxed LP/Recur. rounding ILP (exact) Lower bound Min. vectors CPU s Min. vectors CPU s c432 c432608196.381971.01971.0 c499 c499379260.002601.22602.3 c880 c8801,023125.9712814.0127881.8 c1355 c1355755420.004203.24204.4 c1908 c19081,055543.005434.65436.9 c2670 c2670959477.004774.74777.2 c3540 c35401,971467.2547772.047120008.5 c5315 c53151,079374.3337718.037640.7 c6288 c628824352.525739.05734740.0 c7552 c75522,165841.0084152.0841114.3

36. Copyright Agrawal, 2007ELEC6270 Fall 07, Lecture 836 Finding LP/ILP Solvers R. Fourer, D. M. Gay and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, South San Francisco, California: Scientific Press, 1993. Several of programs described in this book are available to Auburn users. R. Fourer, D. M. Gay and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, South San Francisco, California: Scientific Press, 1993. Several of programs described in this book are available to Auburn users. B. R. Hunt, R. L. Lipsman, J. M. Rosenberg, K. R. Coombes, J. E. Osborn and G. J. Stuck, A Guide to MATLAB for Beginners and Experienced Users, Cambridge University Press, 2006. B. R. Hunt, R. L. Lipsman, J. M. Rosenberg, K. R. Coombes, J. E. Osborn and G. J. Stuck, A Guide to MATLAB for Beginners and Experienced Users, Cambridge University Press, 2006. Search the web. Many programs with small number of variables can be downloaded free. Search the web. Many programs with small number of variables can be downloaded free.