1. Project 2 due … Project 2 due … Project 2 Project 2

2. Chapter 2 Divide and Conquer

3. Recurrence Relations Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer. Ex: Divide and Conquer. T(n) = O(1)if n  c T(n) = a T(n/b) + D(n d ) otherwise Solution Methods Solution Methods Substitution Method. Substitution Method. Recursion-tree Method. Recursion-tree Method. Master Method. Master Method.

4. Divide and Conquer Strategy How does Divide and Conquer Strategy work? How does Divide and Conquer Strategy work? Examples? Examples?

5. Another Example: Merge Sort Sorting Problem: Sort a sequence or n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each Conquer: Sort the two subsequences recursively using merge sort. Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer. Combine: Merge the two sorted subsequences to produce the sorted answer.

6. Merge Sort – Example 4315 9 122261955374399 2 182632 64315 9 122261955374399 2 182632 64315 9 1 22261955374399 2 182632 6 4315 9 1 22261955 374399 2 182632 6 4315 9 1 22261955 374399 2 1826 32 6

7. Merge Sort – Example 182632 64315 9 1 182632 6 4315 9 1 182632 6 4315 9 1 26 18 6 32 15 43 1 9 18 26 32 6 43 15 9 1 182632 64315 9 1 182632 61543 1 9 618 2632 1 91543 1 6 91518263243 1826 18 26 1826 32 6 6 6 182632 6 43 15 4315 9 9 1 1 9 1 4315 9 1 182632 64315 9 1 18 26 6 32 6 2632 18 15 43 1 9 1 9 15 43 1 6 91518 26 32 43 Original SequenceSorted Sequence

8. Analysis of Merge Sort Running time T(n) of Merge Sort: Divide: computing the middle takes O(1) Divide: computing the middle takes O(1) Conquer: solving 2 sub-problems takes 2T(n/2) Conquer: solving 2 sub-problems takes 2T(n/2) Combine: merging n elements takes O(n) Combine: merging n elements takes O(n) Total : Total : T(n) = O(1) if n = 1 T(n) = 2T(n/2) + O(n) if n > 1  T(n) = ?

9. Binary search Algorithm: if (no element in the array), return –1; else else check K against the middle value of the array if (same), return position; else if (K is smaller) binary search the 1 st half of the array; else binary search the 2 st half of the array; T(n) = T(n/2) + 1; T(1) = 1; ? T(n) = ?

10. Multiplication x=10011010, y = 10101011 x=10011010, y = 10101011 x*y = ? – divide and conquer? x*y = ? – divide and conquer?

11. Running Time Recurrence: T(n) = O(1) if n = 1 T(n) = 3T(n/2) + O(n) if n > 1 T(n) = 3T(n/2) + O(n) if n > 1 T(n) = ?

12. Quicksort http://en.wikipedia.org/wiki/Quicksort http://en.wikipedia.org/wiki/Quicksort http://en.wikipedia.org/wiki/Quicksort

13. Recurrence Relations Equation or an inequality that describes a function by its values on smaller inputs. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. Ex: Divide and Conquer. Ex: Divide and Conquer. T(n) = O(1)if n  c T(n) = a T(n/b) + D(n d ) otherwise Solution Methods Solution Methods Substitution Method. Substitution Method. Recursion-tree Method. Recursion-tree Method. Master Method. Master Method.

14. Substitution Method Guess the form of the solution, then use mathematical induction to show it correct. Guess the form of the solution, then use mathematical induction to show it correct. Substitute guessed answer for the function when the inductive hypothesis is applied to smaller values. Substitute guessed answer for the function when the inductive hypothesis is applied to smaller values. Works well when the solution is easy to guess. Works well when the solution is easy to guess. No general way to guess the correct solution. No general way to guess the correct solution.

15. Example –Asymptotics To Solve: T(n) = 2T(n/2) + n Guess: T(n) = O(n lg n) Guess: T(n) = O(n lg n) Need to prove: T(n)  cn lg n, for some c > 0. Need to prove: T(n)  cn lg n, for some c > 0. Hypothesis: T(k)  ck lg k, for all k < n. Hypothesis: T(k)  ck lg k, for all k < n. Calculate: T(n)  2c n/2 lg n/2 + n Calculate: T(n)  2c n/2 lg n/2 + n  c n lg (n/2) + n  c n lg (n/2) + n = c n lg n – c n lg2 + n = c n lg n – c n lg2 + n = c n lg n – n (c lg 2 – 1) = c n lg n – n (c lg 2 – 1)  c n lg n  c n lg n (The last step is true for c  1 / lg2.)

16. Exercises Solution of T(n) = 3T(n/3) + n is Solution of T(n) = 3T(n/3) + n is Solution of T(n) = T(n/3) + 1 is Solution of T(n) = T(n/3) + 1 is Solve T(n) = 2T(n/2) + 1 Solve T(n) = 2T(n/2) + 1 O(nlogn) O(lgn) O(n)O(n)O(n)O(n)

17. Running Time Recurrence: T(n) = O(1) if n = 1 T(n) = 3T(n/2) + O(n) if n > 1 T(n) = 3T(n/2) + O(n) if n > 1 T(n) = ?

18. Recursion-tree Method Making a good guess is sometimes difficult with the substitution method. Making a good guess is sometimes difficult with the substitution method. Use recursion trees to devise good guesses. Use recursion trees to devise good guesses. Recursion Trees Recursion Trees Show successive expansions of recurrences using trees. Show successive expansions of recurrences using trees. Keep track of the time spent on the subproblems of a divide and conquer algorithm. Keep track of the time spent on the subproblems of a divide and conquer algorithm. Help organize the algebraic bookkeeping necessary to solve a recurrence. Help organize the algebraic bookkeeping necessary to solve a recurrence.

19. Recursion Tree – Example Running time of Merge Sort: Running time of Merge Sort: T(n) =  (1) if n = 1 T(n) = 2T(n/2) +  (n) if n > 1 Rewrite the recurrence as Rewrite the recurrence as T(n) = c if n = 1 T(n)  2T(n/2) + cn if n > 1 T(n)  2T(n/2) + cn if n > 1 c > 0: Running time for the base case and time per array element for the divide and time per array element for the divide and combine steps. combine steps.

20. Recursion Tree for Merge Sort For the original problem, we have a cost of cn, plus two subproblems each of size (n/2) and running time T(n/2). cn T(n/2) Each of the size n/2 problems has a cost of cn/2 plus two subproblems, each costing T(n/4). cn cn/2 T(n/4) Cost of divide and merge. Cost of sorting subproblems.

21. Recursion Tree for Merge Sort Continue expanding until the problem size reduces to 1. cn cn/2 cn/4 cccccc lg n cn Total : cnlgn+cn

22. Other Examples Use the recursion-tree method to determine a guess for the recurrences Use the recursion-tree method to determine a guess for the recurrences T(n) = 4T(n/4)+4 T(n) = 4T(n/4)+4 T(n) = T(n/2) + 2T(n/3) + n T(n) = T(n/2) + 2T(n/3) + n

23. The Master Method Based on the Master theorem. Based on the Master theorem. “Cookbook” approach for solving recurrences of the form “Cookbook” approach for solving recurrences of the form T(n) = aT(n/b) + f(n d ) T(n) = aT(n/b) + f(n d ) a  1, b > 1 are constants. a  1, b > 1 are constants. f(n) is asymptotically positive. f(n) is asymptotically positive. n/b does not have to be an integer, but we ignore floors and ceilings. Why? n/b does not have to be an integer, but we ignore floors and ceilings. Why? Requires memorization of three cases. Requires memorization of three cases.

24. Master Theorem

25. Let a  1, b > 1 be constants, f(n) be a function. Let T(n) be defined on nonnegative integers by T(n) = aT(n/b) + f(n), where we can replace n/b by  n/b  or  n/b . Then T(n) can be bounded asymptotically in three cases: 1. 1.If f(n) = O(n log b a–  ) for some  > 0, then T(n) =  ( n log b a ). 2. 2.If f(n) =  (n log b a ), then T(n) =  ( n log b a lg n). 3. 3.If f(n) =  (n log b a+  ) for some constant  > 0, and if, for some constant c < 1 and all sufficiently large n, we have af(n/b)  c f(n), then T(n) =  (f(n)).

26. Master Method – Examples T(n) = 16T(n/4)+n T(n) = 16T(n/4)+n a = 16, b = 4, n log b a = n log 4 16 = n 2. a = 16, b = 4, n log b a = n log 4 16 = n 2. f(n) = n = O(n log b a-  ) = O(n 2-  ), where  = 1  Case 1. f(n) = n = O(n log b a-  ) = O(n 2-  ), where  = 1  Case 1. Hence, T(n) =  (n log b a ) =  (n 2 ). Hence, T(n) =  (n log b a ) =  (n 2 ). T(n) = T(3n/7) + 1 T(n) = T(3n/7) + 1 a = 1, b=7/3, and n log b a = n log 7/3 1 = n 0 = 1 a = 1, b=7/3, and n log b a = n log 7/3 1 = n 0 = 1 f(n) = 1 =  (n log b a )  Case 2. f(n) = 1 =  (n log b a )  Case 2. Therefore, T(n) =  (n log b a lg n) =  (lg n) Therefore, T(n) =  (n log b a lg n) =  (lg n)

27. Master Method – Examples T(n) = 3T(n/2) + n ? T(n) = 3T(n/2) + n ?

28. Exercise Which method? T(n) = 4T(n/3)+4 T(n) = 4T(n/3)+4 T(n) = 2T(n/2 + 17) + n T(n) = 2T(n/2 + 17) + n T(n) = T(n/2) + 2T(n/4) + n T(n) = T(n/2) + 2T(n/4) + n T(n) = T(n-1)+n T(n) = T(n-1)+n

29. Matrix Multiplications

30. Matrix multiplications Example: AB=C Let A,B,C be n  n matrices. What’s the cost to obtain C ? Let A,B,C be n  n matrices. What’s the cost to obtain C ? (assuming n is a power of 2) (assuming n is a power of 2) r = ae + bg s = af + bh t = ce + dg u = cf + dh

31. Cost of the direct method It takes T(n/2) to obtain each of ae, bf, …dh. It takes T(n/2) to obtain each of ae, bf, …dh. It takes n/2  n/2 additions to obtain r, s, t, or u. It takes n/2  n/2 additions to obtain r, s, t, or u. Therefore: Therefore: T(n) = 8T(n/2)+  (n 2 ) = 8T(n/2)+cn 2 = 8T(n/2)+cn 2 = 8( 8T(n/2 2 )+c(n/2) 2 )+cn 2 = 8( 8T(n/2 2 )+c(n/2) 2 )+cn 2 = ? = ?

32. Cost of the direct method It takes T(n/2) to obtain each of ae, bf, …dh. It takes T(n/2) to obtain each of ae, bf, …dh. It takes n/2  n/2 additions to obtain r, s, t, or u. It takes n/2  n/2 additions to obtain r, s, t, or u. Therefore: Therefore: T(n) = 8T(n/2)+  (n 2 ) =  (n 3 ) =  (n 3 )

33. Strassen’s Algorithm Strassen’s Algorithm: an undetermined coefficient method. It’s based on that fact that A+B is much cheaper to calculate than AB. Strassen’s Algorithm: an undetermined coefficient method. It’s based on that fact that A+B is much cheaper to calculate than AB. Outline of the proof: Let P i = A i B i, i=1,…,7 Let P i = A i B i, i=1,…,7 where A i = (  i1 a+  i2 b+  i3 c+  i4 d), B i = (  i1 e+  i2 f+  i3 g+  i4 h),  ij,  ij  {-1,0,1}. B i = (  i1 e+  i2 f+  i3 g+  i4 h),  ij,  ij  {-1,0,1}. Try to determine  ij,  ij such that Try to determine  ij,  ij such that r, s, t, u =   ij P i,  ij  {-1,0,1} r, s, t, u =   ij P i,  ij  {-1,0,1} Note: The hidden coefficients in front of n log 2 7 is larger than the one in front of n 3. Note: The hidden coefficients in front of n log 2 7 is larger than the one in front of n 3.

34. Strassen’s Algorithm Determine the coefficients  ij,  ij : A 1 = a, A 2 = a+b, A 3 = c+d, A 4 = d, A 1 = a, A 2 = a+b, A 3 = c+d, A 4 = d, A 5 = a+d, A 6 = b  d, A 7 = a  c B 1 = f  h, B 2 = h, B 3 = e, B 4 = g  e, B 1 = f  h, B 2 = h, B 3 = e, B 4 = g  e, B 5 = e+h, B 6 = g+h, B 7 = e+f r = P 5 +P 4 -P 2 +P 6 r = P 5 +P 4 -P 2 +P 6 s = P 1 +P 2 t = P 3 +P 4 u = P 5 +P 1  P 3  P 7 Verify u=… Verify u=… Cost for n=8 ?

35. Strassen’s Algorithm Strassen’s Algorithm: T(n) = 7T(n/2)+  (n 2 ) → T(n)=  (n log 2 7 ) = O(n 2.8 )

36. Strassen’s Algorithm n 2.8 n3n3

37. Insertion sort and selection sort Recurrence: T(n) = O(1) if n = 1 T(n) = T(n-1) + O(n) if n > 1 T(n) = ? Complexity? No divide and conquer algorithms