1. Wroclaw University, Sept 18, 2007 1 Approximation via Doubling (Part II) Marek Chrobak University of California, Riverside Joint work with Claire Kenyon-Mathieu

2. Wroclaw University, Sept 18, 2007 2 Doubling method: (for a minimization problem) Choose d 1 < d 2 < d 3 … (typically powers of 2) For j = 1, 2, 3, … Assume that the optimum is ≤ d j Use this bound to construct a solution of cost ≤ C·d j Simple and effective (works for many problems, offline and online) Typically not best possible ratios

3. Wroclaw University, Sept 18, 2007 3 Online Bidding - Reminder Item for sale of value u (unknown to bidder) Buyer bids d 1,d 2,d 3, … until some d j ≥ u Cost: d 1 + d 2 + … + d j Optimum = u Competitive ratio

4. Wroclaw University, Sept 18, 2007 4 Deterministic Bidding - Upper Bound If 2 j-1 < u ≤ 2 j, the ratio is Doubling strategy: bid 1, 2, 4, …, 2 i, …

5. Wroclaw University, Sept 18, 2007 5 Online Bidding Theorem: The optimal competitive ratio for online bidding is: 4 in the deterministic case e  2.72 in the randomized case Randomized e-ing strategy: choose uniformly random x  [0,1), and bid e x, e x+1, e x+2, e x+3, … [folklore] [Chrobak, Kenyon, Noga, Young, ‘06]

6. Wroclaw University, Sept 18, 2007 6 Cow-Path Problem -- Reminder d1d1 d2d2 d3d3 d j+1 0 u d j-1 djdj For d j-1 < u ≤ d j+1 (j odd) 2  bidding ratio extra ratio 1 So the ratio = 2  bidding ratio + 1 = 9 for d j = 2 j

7. Wroclaw University, Sept 18, 2007 7 Theorem: The optimal competitive ratio for the cow- path problem is 9 in the deterministic case  4.59 in the randomized case Solution of (r-1)ln(r-1) = r  2e+1 Connection to online bidding does not work in randomized case -- why? [Gal ‘80] [Baeza-Yates, Culberson, Rawlins ‘93] [Papadimitriou, Yannakakis ‘91] [Kao, Reif, Tate ‘94] …

8. Wroclaw University, Sept 18, 2007 8 Outline: 1. Online bidding 2. Cow-path 3. Incremental medians (size approximation) 4. Incremental medians (cost approximation) 5. List scheduling on related machines 6. Minimum latency tours 7. Incremental clustering

9. Wroclaw University, Sept 18, 2007 9 List Scheduling 1 2 3 4 5 6 7 jobs Given a list of jobs (each with a specified processing time), assign them to processors to minimize makespan (max load) Online algorithm: assignment of a job does not depend on future jobs Goal: small competitive ratio processors time

10. Wroclaw University, Sept 18, 2007 10 1 2 3 4 5 6 7 processors jobs 1 2 3 4 5 6 7 makespan Greedy: Assign each job to the machine with the lightest load

11. Wroclaw University, Sept 18, 2007 11 1 2 3 4 5 6 7 jobs processors 1 2 3 4 5 6 7 makespan better schedule:

12. Wroclaw University, Sept 18, 2007 12 Analysis of Greedy: x = min load before placing last job y = length of last job so greedy’s makespan = x+y ≤ 2 ·optimum makespan x y m machines total load ≥ m·x, so optimum makespan ≥ x optimum makespan ≥ y

13. Wroclaw University, Sept 18, 2007 13 List Scheduling Greedy is (2-1/m)-competitive [Graham ’66] Lower bound ≈1.88 [Rudin III, Chandrasekaran’03] Best known ratio ≈1.92 [Albers ‘99] [Fleischer, Wahl ‘00] Lots of work on randomized algorithms, preemptive scheduling, …

14. Wroclaw University, Sept 18, 2007 14 List Scheduling on Related Machines processors 1 2 3 Related machines: machines may have different speeds  0.25  0.5  1 1 1 jobs 1 1 1

15. Wroclaw University, Sept 18, 2007 15  0.25  0.5  1 1 1 2 3 4 5 6 7 jobs 1 2 3 4 5 6 7 Algorithm 2PACK(L): schedule each job on the slowest machine whose load will not exceed 2L L 2L2L processors 1 2 3 Hey, the opt makespan is at most L

16. Wroclaw University, Sept 18, 2007 16 Lemma: If the little birdie is right (opt makespan ≤ L) then 2PACK will succeed. Proof: Suppose 2PACK fails on job h h’s length on processor 1 ≤ L, so so load of processor 1 > L r = first processor with load ≤ L (or m+1, if no such processor) 1 2 … … m L 2L2L Claim: if opt executes k on a machine in {r,r+1,…,m} then so does 2PACK optimum 2PACK r r

17. Wroclaw University, Sept 18, 2007 17 1 2 … … m L 2L2L k so k‘s length here ≤ L so k fits on r k r r optimum 2PACK k suppose k executed here Lemma: If the little birdie is right (opt makespan ≤ L) then 2PACK will succeed. Proof: Suppose 2PACK fails on job h h’s length on processor 1 ≤ L, so so load of processor 1 > L r = first processor with load ≤ L (or m+1, if no such processor) Claim: if opt executes k on a machine in {r,r+1,…,m} then so does 2PACK

18. Wroclaw University, Sept 18, 2007 18 1 2 … … m L 2L2L r r optimum 2PACK So opt’s (speed-weighted) total load on processors {1,2,…,r-1} is > (r-1)L Lemma: If the little birdie is right (opt makespan ≤ L) then 2PACK will succeed. Proof: Suppose 2PACK fails on job h h’s length on processor 1 ≤ L, so so load of processor 1 > L r = first processor with load ≤ L (or m+1, if no such processor) In other words: if 2PACK executes k on a machine in {1,2,…,r-1} then so does opt So some opt’s processor has load > L -- contradiction

19. Wroclaw University, Sept 18, 2007 19 Algorithm: 1. Choose d 1 < d 2 < d 3 < … (makespan estimates) Let B j = 2·( d 1 + d 2 + … + d j ) “bucket” j : time interval [B j-1, B j ] 2. j = 0 while there are unassigned jobs apply 2PACK with L = d j in bucket j if 2PACK fails on job k let j = j+1 and continue (starting with job k)

20. Wroclaw University, Sept 18, 2007 20 k bucket j 1 2 m … processor B1B1 B2B2 B j-1 BjBj B j+1 … k k’

21. Wroclaw University, Sept 18, 2007 21 Analysis: Suppose the optimal makespan is u Choose j such that d j-1 < u ≤ d j Then 2PACK will succeed in j ’th bucket (L = d j ) so algorithm’s makespan ≤ 2·(d 1 +d 2 + … + d j ) and We get ratio 8 for d j = 2 j 2  (bidding ratio)

22. Wroclaw University, Sept 18, 2007 22 Theorem: There is an 8-competitive online algorithm for list scheduling on related machines (to minimize makespan). With randomization the ratio can be improved to 2e. [Aspnes, Azar, Fiat, Plotkin, Waarts ‘06] World records: upper bound ≈ 5.828 (4.311 randomized) lower bound ≈ 2.438 (2 randomized) [Berman, Charikar, Karpinski ‘97] [Epstein, Sgall ‘00] List Scheduling on Related Machines

23. Wroclaw University, Sept 18, 2007 23 Outline: 1. Online bidding 2. Cow-path 3. Incremental medians (size approximation) 4. Incremental medians (cost approximation) 5. List scheduling on related machines 6. Minimum latency tours 7. Incremental clustering

24. Wroclaw University, Sept 18, 2007 24 Minimum Latency Tour X = metric space P = v 1 v 2 …v h : path in X Latency of v i on P latency P (v i ) = d(v 1, v 2 ) + … + d(v i-1, v i ) (Total) latency of P =  i latency P (v i ) Minimum Latency Tour Problem: Given X, find a tour (path visiting all vertices) of minimum total latency Goal: polynomial-time approximation algorithm

25. Wroclaw University, Sept 18, 2007 25 Total latency = 2 + 4 + 8 + 11 + 15 = 40 A 15 E 2 D 11 C 4 B 8 F

26. Wroclaw University, Sept 18, 2007 26 A E D C B F Minimum k-Tour Problem: find a shortest k-tour (a path that starts and ends at v 1 and visits ≥ k different vertices) 2-tour 4-tour

27. Wroclaw University, Sept 18, 2007 27 Algorithm: 1. Choose d 1 < d 2 < d 3 < … 2. For each k compute the optimal k-tour T k 3. Choose p(1) < … < p(m) = n s.t. length(T p(i) ) = d i (For simplicity assume they exist) 4. Output Q = T p(1) T p(2) …T p(m) (concatenation) Denote Q = q 1 q 2 …q n (q i = first point on Q different from q 1, q 2,…,q i-1 )

28. Wroclaw University, Sept 18, 2007 28 v1v1 T p(1) T p(2) T p(3) Q

29. Wroclaw University, Sept 18, 2007 29 Lemma: S = s 1 s 2 …s n : tour with optimum latency. Then latency S (s k ) ≥ (1/2) · length(T k ) Proof: s1s1 s2s2 s3s3 sksk S T T is a k-tour, so 2·latency S (s k ) = length(T) ≥ length(T k )

30. Wroclaw University, Sept 18, 2007 30 Analysis: For p(j-1) < k ≤ p(j) latency S (s k ) ≥ (1/2)·length(T k ) ≥ (1/2)·length(T p(j-1) ) = d j-1 /2 q k will be visited in T p(j) (or earlier), so latency Q (q k ) ≤ d 1 +d 2 + … + d j We get ratio 8 for d j = 2 j … if we can compute k-tours efficiently !!! 2  (bidding ratio)

31. Wroclaw University, Sept 18, 2007 31 If X is a weighted tree, optimal k-tours can be computed in polynomial time… 10 4 5 11 12 7 6 8 2 3 9 7 7 5 Theorem: There is a polynomial-time 8-approximation algorithm for maximum latency tours on weighted trees [Blum, Chalasani, Coppersmith, Pulleyblank, Raghavan, Sudan ‘94] Dynamic programming: W.l.o.g. assume X is a rooted binary tree opt k (u) = minimum of 2x+opt k-1 (v), 2y+opt k-1 (t) and min j {2x+opt j (v)+2y+opt k-1-j (t) } u v t x y

32. Wroclaw University, Sept 18, 2007 32 Choose a random direction (clockwise or counter-clockwise) and traverse each T p(j) in this direction … v1v1 Tp(j)Tp(j) u Expected latency of u = d 1 +d 2 + …+ d j-1 + d j /2 We get ratio 6 for d j = 2 j Can we do better?

33. Wroclaw University, Sept 18, 2007 33 Can be extended to arbitrary spaces, with ratio 3.591 [Chauduri, Godfrey, Rao, Talwar ‘03] Theorem: There is a polynomial-time 3.591 -approximation algorithm for maximum latency tours on weighted trees [Goemans, Kleinberg ‘98] Can we do even better? Instead of d j = 2 j choose d j = c j+x, where c is the constant from the Cow Path problem and x is random in [0,1) We don’t really really randomization: choose better direction (clockwise or counter-clockwise) There are only O(n) x’s that matter, so try them all

34. Wroclaw University, Sept 18, 2007 34 Outline: 1. Online bidding 2. Cow-path 3. Incremental medians (size approximation) 4. Incremental medians (cost approximation) 5. List scheduling on related machines 6. Minimum latency tours 7. Incremental clustering

35. Wroclaw University, Sept 18, 2007 35 k-Clustering X = metric space For C  X, diameter(C) = maximum distance between points in C k-Clustering Problem: Given k, partition X into k disjoint clusters C 1,…,C k to minimize the maximum diameter(C j ) Offline: approximable with ratio 2 [Gonzales ‘85] [Hochbaum, Shmoys ‘85] lower bound of 2 for polynomial algorithms (unless P = NP) [Feder, Greene ‘88] [Bern, Eppstein ‘96]

36. Wroclaw University, Sept 18, 2007 36 E D C B F A G H 3-Clustering with maximum diameter 5 k=3

37. Wroclaw University, Sept 18, 2007 37 E D C B F A G H 3-Clustering with maximum diameter 3 k=3

38. Wroclaw University, Sept 18, 2007 38 Incremental k-Clustering Problem: Maintain k-clustering when points in X arrive online allowed operations:  add point to a cluster  merge clusters  create a new singleton cluster Goal: online competitive algorithm (polynomial-time) different model than incremental medians !!!

39. Wroclaw University, Sept 18, 2007 39 D C G diameter = 0 A k=3

40. Wroclaw University, Sept 18, 2007 40 D C G A E diameter = 2 k=3

41. Wroclaw University, Sept 18, 2007 41 D C G A E H diameter = 3 k=3

42. Wroclaw University, Sept 18, 2007 42 D C G A E H diameter = 3 k=3

43. Wroclaw University, Sept 18, 2007 43 Notation and terminology: Algorithm’s clusters C 1,C 2,…,C k’ with k’ ≤ k in each C i fix a center o i radius of C i = max distance between x  C i and o i diameter of C i ≤ 2 · (radius of C i ) CiCi oioi radius

44. Wroclaw University, Sept 18, 2007 44 Procedure CleanUp(z). Goal: merge some clusters C 1,C 2,…,C k’ so that afterwards all inter-center distances are > z 1.Find a maximal set J of clusters with all inter- center distances > z 2. for each cluster C a  J choose C b  J with d(o a,o b ) ≤ z merge C a into C b (with center o b )

45. Wroclaw University, Sept 18, 2007 45 Lemma: If the max radius before CleanUp is h then after CleanUp it is ≤ h+z. Proof: follows from the ∆ inequality z

46. Wroclaw University, Sept 18, 2007 46 Lemma: If the max radius before CleanUp is h then after CleanUp it is ≤ h+z. Proof: follows from the ∆ inequality z v z h

47. Wroclaw University, Sept 18, 2007 47 Algorithm: 1. Choose d 1 < d 2 < d 3 < … 2. Initially C 1,C 2,…,C k are singletons (first k points) (Assume min distance between these points is > 1) 3. j  1 4. repeat when a new point x arrives if d(x,o i ) ≤ d j for some i add x to C i else if k’ < k k’  k’+1; C k’  {x} else create a temporary cluster C k+1  {x} while there k+1 clusters j  j+1 do CleanUp with z = d j (merge clusters) checkpoint j Invariant: inter-center distance > d j

48. Wroclaw University, Sept 18, 2007 48 Example: k = 4

49. Wroclaw University, Sept 18, 2007 49 Example: k = 4 d j+1 d j+2

50. Wroclaw University, Sept 18, 2007 50 Analysis: At checkpoint j : Before clean-up k+1 clusters with inter-center distances > d j-1 so opt diameter > d j-1 After clean-up max radius ≤ d 1 + d 2 + … + d j so max diameter ≤ 2·(d 1 + d 2 + … + d j ) We get ratio 8 for d j = 2 j 2  (bidding ratio)

51. Wroclaw University, Sept 18, 2007 51 Theorem: There is an 8-competitive online algorithm for incremental clustering (ratio 2e with randomization). [Charikar, Chekuri, Feder, Motwani ‘06] Other results: upper bound 6 (4.33 randomized) (not polynomial-time) lower bound 2.414 (2 randomized) Incremental Clustering

52. Wroclaw University, Sept 18, 2007 52 1. Online bidding 2. Cow-path 3. Incremental medians (size approximation) 4. Incremental medians (cost approximation) 5. List scheduling on related machines 6. Minimum latency tours 7. Incremental clustering 8. Other scheduling problems: List scheduling, related machines with preemption Scheduling with min-sum criteria Non-clairvoyant scheduling 9. Hierarchical clustering 10. Load balancing 11. Online algorithms for set cover (combined with primal-dual) …. Doubling Method Applications:

53. Wroclaw University, Sept 18, 2007 53 Thank you ! Questions?