1. 1 Prune-and-Search Method 2012/10/30

2. A simple example: Binary search sorted sequence : (search 9) 14579101215 step 1  step 2  step 3  Binary search can be regarded as a case of prune-and-search method. After each comparison, a half of the data set are pruned away. Binary search can be viewed as a special divide- and-conquer method, since there exists no solution in another half and then no merging is done.

3. 3 Time complexity: Assume n = 2 k, T(n)= T(n/2)+1 = T(n/4)+1+1 : =T(n/2 k )+k*1 =k =log n Binary Search

4. The selection problem Input: A set S of n elements Output: The k-th smallest element of S The median problem: to find the -th smallest element. The straightforward algorithm: step 1: Sort the n elements step 2: Locate the k-th element in the sorted list. Time complexity: O(n log n)

5. The selection problem Prune-and-search S={a 1, a 2, …, a n } Let p  S, use p to partition S into 3 subsets S 1, S 2, S 3 : S 1 ={ a i | a i < p, 1  i  n} S 2 ={ a i | a i = p, 1  i  n} S 3 ={ a i | a i > p, 1  i  n} If |S 1 | > k, then the k-th smallest element of S is in S 1, prune away S 2 and S 3. Else, if |S 1 | + |S 2 | > k, then p is the k-th smallest element of S. Else, the k-th smallest element of S is the (k - |S 1 | - |S 2 |)-th smallest element in S 3, prune away S 1 and S 2.

6. The selection problem How to select P? The n elements are divided into subsets. (Each subset has 5 elements.) M P At least 1/4 of S known to be less than or equal to P. Each 5-element subset issorted in non-decreasingsequence. At least 1/4 of S known to be greater than or equal to P.

7. Prune-and-Search approach Input: A set S of n elements. Output: The k-th smallest element of S. Step 1. Divide S into  n/5  subsets. Each subset contains five elements. Add some dummy  elements to the last subset if n is not a net multiple of 5. Step 2. Sort each subset of elements directly. Step 3. Find the element p which is the median of the medians of the  n/5  subsets.

8. Prune-and-Search approach Step 4. Partition S into S 1, S 2 and S 3, which contain the elements less than, equal to, and greater than p, respectively Step 5. If |S 1 |  k, then discard S 2 and S 3 and solve the problem that selects the k-th smallest element from S 1 during the next iteration; else if |S 1 | + |S 2 |  k then p is the k-th smallest element of S; otherwise, let k = k - |S 1 | - |S 2 |, solve the problem that selects the k’-th smallest element from S 3 during the next iteration.

9. Prune-and-Search approach At least n/4 elements are pruned away during each iteration. The problem remaining in step 5 contains at most 3n/4 elements. Time complexity: T(n) = O(n) step 1: O(n) step 2: O(n) step 3: T(n/5) step 4: O(n) step 5: T(3n/4) T(n) = T(3n/4) + T(n/5) + O(n)

10. Prune-and-Search approach Let T(n) = a 0 + a 1 n + a 2 n 2 + …, a 1  0 T(3n/4) = a 0 + (3/4)a 1 n + (9/16)a 2 n 2 + … T(n/5) = a 0 + (1/5)a 1 n + (1/25)a 2 n 2 + … T(3n/4 + n/5) = T(19n/20) = a 0 + (19/20)a 1 n + (361/400)a 2 n 2 + … T(3n/4) + T(n/5)  a 0 + T(19n/20)  T(n)  cn + T(19n/20)  cn + (19/20)cn +T((19/20) 2 n) …  cn + (19/20)cn + (19/20) 2 cn + … +(19/20) p cn + T((19/20) p+1 n), (19/20) p+1 n  1  (19/20) p n =  20 cn +b = O(n)

11. 11 The General Prune-and-Search Method It consists of many iterations. At each iteration, it prunes away a fraction, say f, of the input data, and then it invokes the same algorithm recursively to search the solution for the remaining data. After some iterations, the size of input data will be small enough that the problem can be solved directly in some constant time c.

12. 12 The General Method Time-complexity analysis : Assume that the time needed to execute the prune-and-search in each iteration is O(n k ) for some constant k >0 and the worst case time complexity of the prune- and-search algorithm is T(n). Then T(n) = T((1  f ) n) + O(n k )

13. 13 The General Method We have T(n)  T((1   f ) n) + cn k for sufficiently large n.  T((1   f ) 2 n) + cn k + c(1   f ) k n k …  c ’ + cn k + c(1   f ) k n k + c(1   f ) 2k n k +... + c(1   f ) pk n k = c ’ + cn k (1 + (1   f ) k + (1   f ) 2k +... + (1   f ) pk ). Since 1   f < 1, as n ,  T(n) = O(n k ) Thus, the time-complexity of the whole prune- and-search process is of the same order as the time-complexity in each iteration Thus, the time-complexity of the whole prune- and-search process is of the same order as the time-complexity in each iteration

14. Linear Programming (Linear Optimization) 14 subject to: Maximize or minimize

15. Feasible Region Example in 2D optimum basic constraints max x 1 + 8x 2 subject to: (1) x 1  3 (2) x 2  2 (3)–3x 1 + 4x 2  14 (4) 4x 1 – 3x 2  25 (5) x 1 + x 2  15 x2x2 x1x1 (1) (2) (4) (5) (3) (5) (3) optimum x 1 + 8x 2 x 1 = 46/7 x 2 = 59/7 = 74

16. Example in 3D subject to: maximize z Optimum (x,y,z)=(0,0,3) x y z

17. 6 -17 Linear programming with two variables Minimize ax + by subject to a i x + b i y  c i, i = 1, 2, …, n Simplified two-variable linear programming problem: Minimize y subject to y  a i x + b i, i = 1, 2, …, n

18. 6 -18 The boundary F(x): F(x) = The optimum solution x 0 : F(x 0 ) = F(x) x y F(x) a 4 x + b 4 a 2 x + b 2 a 3 x + b 3 a 5 x + b 5 a 7 x + b 7 a 6 x + b 6 a 1 x + b 1 a 8 x + b 8 (x 0,y 0 )

19. 6 -19 Constraints deletion If x 0 < x m and the intersection of a 3 x + b 3 and a 2 x + b 2 is greater than x m, then one of these two constraints is always smaller than the other for x < x m. Thus, this constraint can be deleted. It is similar for x 0 > x m. x y a 8 x + b 8 x0x0 xmxm a 1 x + b 1 a 2 x + b 2 a 4 x + b 4 a 3 x + b 3 a 5 x + b 5 a 7 x + b 7 a 6 x + b 6 May be deleted

20. 6 -20 Determining the direction of the optimum solution Let y m = F(x m ) = Case 1: y m is on only one constraint. Let g denote the slope of this constraint. If g > 0, then x 0 < x m. If g x m. y x y'my'm ymym xmxm x'mx'm x0x0 x0x0 The cases where x m is on only one constrain. Suppose an x m is known. How do we know whether x 0 x m ?

21. 6 -21 Case 2: y m is the intersection of several constraints. g max = max. slope g min = min. slop If g min > 0, g max > 0, then x 0 < x m If g min x m If g min 0, then (x m, y m ) is the optimum solution. y x x m,3 x m,2 x m,1 Case 2a: x m,3 Case 2b: x m,1 Case 2c: x m,2 g max g min g max g min Cases of x m on the intersection of several constraints.

22. 6 -22 How to choose x m ? We arbitrarily group the n constraints into n/2 pairs. For each pair, find their intersection. Among these n/2 intersections, choose the median of their x-coordinates as x m.

23. 6 -23 Prune-and-Search approach Input: Constraints S: a i x + b i, i=1, 2, …, n. Output: The value x 0 such that y is minimized at x 0 subject to the above constraints. Step 1: If S contains no more than two constraints, solve this problem by a brute force method. Step 2: Divide S into n/2 pairs of constraints randomly. For each pair of constraints a i x + b i and a j x + b j, find the intersection p ij of them and denote its x-value as x ij. Step 3: Among the x ij ’s, find the median x m.

24. 6 -24 Step 4: Determine y m = F(x m ) = g min = g max = Step 5: Case 5a: If g min and g max are not of the same sign, y m is the solution and exit. Case 5b: otherwise, x 0 0, and x 0 >x m, if g min < 0.

25. 6 -25 Step 6: Case 6a: If x 0 < x m, for each pair of constraints whose x-coordinate intersection is larger than x m, prune away the constraint which is always smaller than the other for x  x m. Case 6b: If x 0 > x m, do similarly. Let S denote the set of remaining constraints. Go to Step 2. There are totally  n/2  intersections. Thus,  n/4  constraints are pruned away for each iteration. Time complexity: T(n) = T(3n/4)+O(n) = O(n)

26. 26 The 1-center problem Given n planar points, find a smallest circle to cover these n points. The 1-Center Problem

27. 27 The constrained 1-center problem The center is restricted to lying on a straight line, say y=y’=0.

28. 28 Prune-and-Search approach Input : n points and a straight line y = y’. Output : The constrained center on the straight line y = y’. Step 1. If n is no more than 2, solve this problem by a brute-force method. Step 2. Form disjoint pairs of points (p 1, p 2 ), (p 3, p 4 ), …,(p n-1, p n ). If there are odd number of points, just let the final pair be (p n, p 1 ). Step 3. For each pair of points, (p i, p i+1 ), find the point x i,i+1 on the line y = y’ such that d(p i, x i,i+1 ) = d(p i+1, x i,i+1 ).

29. 29 Prune-and-Search approach Step 4. Find the median of the x i,i+1 ’s. Denote it as x m. Step 5. Calculate the distance between p i and x m for all i. Let p j be the point which is farthest from x m. Let x j denote the projection of p j onto y = y’. If x j is to the left (right) of x m, then the optimal solution, x *, must be to the left (right) of x m. Step 6. If x * x m, prune the point p i if p i is closer to x m than p i+1, otherwise prune the point p i+1 ; If x * > x m, for each x i,i+1 < x m, prune the point p i if p i is closer to x m than p i+1 ; otherwise prune the point p i+1. Step 7. Go to Step 1.

30. 30 Prune-and-Search approach y = y’ L ij PiPi P i+1 X i,i+1 xmxm x* PjPj The Pruning of Points in the Constrained 1-Center Problem

31. 31 Prune-and-Search approach Since there are x i,i+1 ’s lying in the left (right) side of x m, we can prune away points for each iteration of the algorithm. Each iteration takes O(n) time. Hence the time-complexity of this algorithm is T(n)=T(3n/4)+O(n)=O(n), as n   Time complexity: O(n)

32. 32 Q&A