1. Polynomial time approximation scheme Lecture 17: Mar 13

2. Polynomial Time Approximation Scheme (PTAS) We have seen the definition of a constant factor approximation algorithm. The following is something even better. A An algorithm A is an approximation scheme if for every є > 0, A A runs in polynomial time (which may depend on є) and return a solution: SOL ≤ (1+є)OPT for a minimization problem SOL ≥ (1-є)OPT for a maximization problem A For example, A may run in time n 100/є. There is a time-accuracy tradeoff.

3. Knapsack Problem A set of items, each has different size and different value. We only have one knapsack. Goal: to pick a subset which can fit into the knapsack and maximize the value of this subset.

4. Knapsack Problem (The Knapsack Problem) Given a set S = {a 1, …, a n } of objects, with specified sizes and profits, size(a i ) and profit(a i ), and a knapsack capacity B, find a subset of objects whose total size is bounded by B and total profit is maximized. Assume size(a i ), profit(a i ), and B are all integers. We’ll design an approximation scheme for the knapsack problem.

5. Greedy Methods Sort by object size in non-decreasing order: General greedy method: Sort the objects by some rule, and then put the objects into the knapsack according to this order. Sort by profit in non-increasing order: Sort by profit/object size in non-increasing order: Greedy won’t work.

6. Exhaustive Search n objects, total 2 n possibilities, view it as a search tree. choose object 1 not choose object 1 choose object 2not choose object 2choose object 2not choose object 2 At the bottom we could calculate the total size and total profit, and choose the optimal subset.

7. Exhaustive Search size(a1)=2, profit(a1)=4 size(a2)=3, profit(a2)=5 size(a3)=2, profit(a3)=3 size(a4)=1, profit(a4)=2 (0,0) (Total size, total profit) choose object 1 not choose object 1 (2,4)(0,0) (5,9)(2,4) (4,7)(5,9)(7,12) (3,5) (5,8)(2,3) There are many redundancies.

8. Exhaustive Search size(a1)=2, profit(a1)=4 size(a2)=3, profit(a2)=5 size(a3)=2, profit(a3)=3 size(a4)=1, profit(a4)=2 (0,0) (Total size, total profit) choose object 1 not choose object 1 (2,4)(0,0) (5,9)(2,4) (4,7)(5,9)(7,12) (3,5) (5,8)(2,3) There are many redundancies. (4,7)(5,9) (6,11)(4,7)(3,5)

9. Exhaustive Search size(a1)=2, profit(a1)=4 size(a2)=3, profit(a2)=5 size(a3)=2, profit(a3)=3 size(a4)=1, profit(a4)=2 (0,0) (Total size, total profit) choose object 1 not choose object 1 (2,4)(0,0) (5,9)(2,4) (4,7)(5,9)(7,12) (3,5) (5,8)(2,3) There are many redundancies. (4,7)(5,9) (6,11)(4,7)(3,5)

10. The Idea Consider two subproblems P and Q at the same level (i.e. same number of objects have been considered).  If size(P)=size(Q) and profit(P)=profit(Q), just compute either one.  If size(P)=size(Q) and profit(P)>profit(Q), just compute P.  If profit(P)=profit(Q) and size(P)>size(Q), just compute Q. Important: the history doesn’t matter (i.e. which subset we chose to achieve profit(P) and size(P)).

11. Dynamic Programming Dynamic programming is just exhaustive search with polynomial number of subproblems. We only need to compute each subproblem once, and each subproblem is looked up at most a polynomial number of times, and so the total running time is at most a polynomial.

12. Dynamic Programming for Knapsack Suppose we have considered object 1 to object i. We want to remember what profits are achievable. For each achievable profit, we want to minimize the size. Let S(i,p) denote a subset of {a1,…,ai} whose total profit is exactly p and total size is minimized. Let A(i,p) denote the size of the set S(i,p) (A(i,p) = ∞ if no such set exists). For example, A(1,p) = size(a1) if p=profit(a1), Otherwise A(1,p) = ∞ (if p ≠ profit(a1)).

13. Recurrence Formula How to compute A(i+1,p) if we know A(i,q) for all q? Idea: we either choose object i+1 or not. If we do not choose object i+1: then A(i+1,p) = A(i,p). Remember: A(i,p) denote the minimize size to achieve profit p using objects from 1 to i. If we choose object i+1: then A(i+1,p) = size(a i+1 ) + A(i,p-profit(a i+1 )) if p > profit(a i+1 ). A(i+1,p) is the minimum of these two values.

14. size(a1)=2, profit(a1)=4; size(a2)=3, profit(a2)=5; size(a3)=2, profit(a3)=3; size(a4)=1, profit(a4)=2 An Example Optimal Solution: max{ p | A(n,p) ≤ B} where B is the size of the knapsack. 01234567891011121314 0∞∞∞2∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ Remember: A(i,p) denote the minimize size to achieve profit p using objects from 1 to i. 1 2 3 4

15. size(a1)=2, profit(a1)=4; size(a2)=3, profit(a2)=5; size(a3)=2, profit(a3)=3; size(a4)=1, profit(a4)=2 An Example A(i+1,p) = min{A(i,p), size(a i+1 ) + A(i,p-profit(a i+1 ))}. A(2,p) = min{A(1,p), A(1,p-5)+3}. 1 2 3 4 01234567891011121314 0∞∞∞2∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞

16. size(a1)=2, profit(a1)=4; size(a2)=3, profit(a2)=5; size(a3)=2, profit(a3)=3; size(a4)=1, profit(a4)=2 An Example A(i+1,p) = min{A(i,p), size(a i+1 ) + A(i,p-profit(a i+1 ))}. 01234567891011121314 0∞∞∞2∞∞∞∞∞∞∞∞∞∞ 0∞∞∞23∞∞∞5∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 1 2 3 4 A(3,p) = min{A(2,p), A(2,p-3)+2}.

17. size(a1)=2, profit(a1)=4; size(a2)=3, profit(a2)=5; size(a3)=2, profit(a3)=3; size(a4)=1, profit(a4)=2 An Example A(i+1,p) = min{A(i,p), size(a i+1 ) + A(i,p-profit(a i+1 ))}. 01234567891011121314 0∞∞∞2∞∞∞∞∞∞∞∞∞∞ 0∞∞∞23∞∞∞5∞∞∞∞∞ 0∞∞223∞455∞∞7∞∞ 0∞∞∞∞∞∞∞∞∞∞∞∞∞∞ 1 2 3 4 A(4,p) = min{A(3,p), A(3,p-2)+1}.

18. size(a1)=2, profit(a1)=4; size(a2)=3, profit(a2)=5; size(a3)=2, profit(a3)=3; size(a4)=1, profit(a4)=2 An Example A(i+1,p) = min{A(i,p), size(a i+1 ) + A(i,p-profit(a i+1 ))}. A(4,p) = min{A(3,p), A(3,p-2)+1}. 01234567891011121314 0∞∞∞2∞∞∞∞∞∞∞∞∞∞ 0∞∞∞23∞∞∞5∞∞∞∞∞ 0∞∞223∞455∞∞7∞∞ 0∞12233455667∞8 1 2 3 4

19. An Example 01234567891011121314 0∞∞∞2∞∞∞∞∞∞∞∞∞∞ 0∞∞∞23∞∞∞5∞∞∞∞∞ 0∞∞223∞455∞∞7∞∞ 0∞12233455667∞8 1 2 3 4 Optimal Solution: max{ p | A(n,p) ≤ B} where B is the size of the knapsack. Remember: A(i,p) denote the minimize size to achieve profit p using objects from 1 to i. For example, if B=8, OPT=14, if B=7, OPT=12, if B=6, OPT=11.

20. Running Time For the dynamic programming algorithm, there are n rows and at most nP columns. Each entry can be computed in constant time (look up two entries). So the total time complexity is O(n 2 P). The input has 2n numbers, say each is at most P. So the input has total length 2nlog(P). The running time is not polynomial if P is very large (compared to n).

21. Approximation Algorithm We know that the knapsack problem is NP-complete. Can we use the dynamic programming technique to design approximation algorithm?

22. Scaling Down Idea: to scale down the numbers and compute the optimal solution in this modified instance  Suppose P ≥ 1000n.  Then OPT ≥ 1000n.  Now scale down each element by 100 times (profit*:=profit/100).  Compute the optimal solution using this new profit.  Can’t distinguish between element of size, say 2199 and 2100.  Each element contributes at most an error of 100.  So total error is at most 100n.  This is at most 1/10 of the optimal solution.  However, the running time is 100 times faster.

23. Approximation Scheme Goal: to find a solution which is at least (1- є)OPT for any є > 0. Approximation Scheme for Knapsack 1.Given є > 0, let K = єP/n, where P is the largest profit of an object. 2.For each object ai, define profit*(ai) = profit(ai)/K. 3.With these as profits of objects, using the dynamic programming algorithm, find the most profitable set, say S’. 4.Output S’ as the approximate solution.

24. Quality of Solution Theorem. Let S denote the set returned by the algorithm. Then, profit(S) ≥ (1- є)OPT. Proof. Let O denote the optimal set. For each object a, because of rounding down, K·profit*(a) can be smaller than profit(a), but by not more than K. Since there are at most n objects in O, profit(O) – K·profit*(O) ≤ nK. Since the algorithm return an optimal solution under the new profits, profit(S) ≥ K·profit*(S) ≥ K·profit*(O) ≥ profit(O) – nK = OPT – єP ≥ (1 – є)OPT because OPT ≥ P.

25. Running Time For the dynamic programming algorithm, there are n rows and at most n P/K columns. Each entry can be computed in constant time (look up two entries). So the total time complexity is O(n 2 P/K ) = O(n 3 / є). Therefore, we have an approximation scheme for Knapsack.

26. Approximation Scheme Quick Summary 1.Modify the instance by rounding the numbers. 2.Use dynamic programming to compute an optimal solution S in the modified instance. 3.Output S as the approximate solution.

27. Bin Packing (Bin Packing) Given n items with sizes 0<= a1,a2,…,an <= 1, find a packing in unit-sized bins that minimizes the number of bins used. e.g. Paper cutting. Greedy algorithm gives a 2-approximation. Theorem. For any 0 < є <= 1/2, there is a polynomial time algorithm, which finds a packing using at most (1+2є)OPT + 1 bins.

28. Exhaustive Search How do you solve the problem optimally? Suppose each bin can pack at most M items, and there are only K different item sizes. What is the running time of this algorithm? Try all possible bin configuration, and then try all possible combination of bins.

29. Counting Doughnut Selections There are five kinds of doughnuts. How many different ways to select a dozen doughnuts? A ::= all selections of a dozen doughnuts Hint: define a bijection! 00 (none) 000000 00 00 Chocolate Lemon Sugar Glazed Plain

30. Counting Doughnut Selections A ::= all selections of a dozen doughnuts Define a bijection between A and B. 00 1 1 000000 1 00 1 00 Each doughnut is represented by a 0, and four 1’s are used to separate five types of doughnuts. B::= all 16-bit binary strings with exactly four 1’s. 00 (none) 000000 00 00 Chocolate Lemon Sugar Glazed Plain

31. Counting Doughnut Selections c chocolate, l lemon, s sugar, g glazed, p plain maps to 0 c 10 l 10 s 10 g 10 p B::= all 16-bit binary strings with exactly four 1’s. A ::= all selections of a dozen doughnuts

32. Exhaustive Search Suppose each bin can pack at most M items, and there are only K different item sizes. What is the running time of this algorithm? At mostbin configurations At most combinations of bins! This is polynomial time if M and K are constants!

33. Throw away all small items of size at most є. Reduction 1 How to make sure that each bin uses at most M items? Suppose there is a (1+є)-approximation when there are no small items, then we can finish the packing with at most (1+2є)OPT+1 bins. Pack all the small items into the remaining space, and open new bins if necessary. Let M be the number of bins used. Then OPT >= (M-1)(1 - є).

34. How to make sure that there are at most K distinct item sizes? Reduction 2 Round the item size! Round up Round down Maintain feasibility but may use more bins Will not use more bins but may not be feasible

35. Reduction 2 Round up Round down Maintain feasibility but may use more bins Will not use more bins but may not be feasible Prove that the “round up” solution is not much worse than OPT by comparing it to the “round down” solution. K groups, each of size n/K

36. Reduction 2 Round up Round down Maintain feasibility but may use more bins Will not use more bins but may not be feasible Suppose there is a feasible “round down” solution, construct an almost feasible “round up” solution with the last n/K items not packed. K groups, each of size n/K

37. Reduction 2 Round up Round down Maintain feasibility but may use more bins Will not use more bins but may not be feasible Then n/K <= n є 2 <= єOPT In the worst case we use n/K bins more than the optimal. Set K = 1/є 2

38. 1.Remove small items of size < є Round up to obtain constant (1/є 2 ) number of item sizes (Reduction 2) Find optimal packing (exhaustive search, doughnuts) Use this packing for original item sizes Pack small items back greedily (Reudction 1) Algorithm

39. Minimum makespan scheduling Euclidean TSP Euclidean minimum Steiner tree PTAS But most problems do not admit a PTAS unless P=NP. Project proposal: due March 20 Sign up for meeting.