1. Divide And Conquer Distinguish between small and large instances. Small instances solved differently from large ones.

2. Small And Large Instance Small instance.  Sort a list that has n <= 10 elements.  Find the minimum of n <= 2 elements. Large instance.  Sort a list that has n > 10 elements.  Find the minimum of n > 2 elements.

3. Solving A Small Instance A small instance is solved using some direct/simple strategy.  Sort a list that has n <= 10 elements. Use count, insertion, bubble, or selection sort.  Find the minimum of n <= 2 elements. When n = 0, there is no minimum element. When n = 1, the single element is the minimum. When n = 2, compare the two elements and determine which is smaller.

4. Solving A Large Instance A large instance is solved as follows:  Divide the large instance into k >= 2 smaller instances.  Solve the smaller instances somehow.  Combine the results of the smaller instances to obtain the result for the original large instance.

5. Sort A Large List Sort a list that has n > 10 elements.  Sort 15 elements by dividing them into 2 smaller lists.  One list has 7 elements and the other has 8.  Sort these two lists using the method for small lists.  Merge the two sorted lists into a single sorted list.

6. Find The Min Of A Large List Find the minimum of 20 elements.  Divide into two groups of 10 elements each.  Find the minimum element in each group somehow.  Compare the minimums of each group to determine the overall minimum.

7. Recursion In Divide And Conquer Often the smaller instances that result from the divide step are instances of the original problem (true for our sort and min problems). In this case,  If the new instance is a small instance, it is solved using the method for small instances.  If the new instance is a large instance, it is solved using the divide-and-conquer method recursively. Generally, performance is best when the smaller instances that result from the divide step are of approximately the same size.

8. Recursive Find Min Find the minimum of 20 elements.  Divide into two groups of 10 elements each.  Find the minimum element in each group recursively. The recursion terminates when the number of elements is <= 2. At this time the minimum is found using the method for small instances.  Compare the minimums of the two groups to determine the overall minimum.

9. Divide-And-Conquer Sorting Small instance.  n <= 1 elements.  n <= 10 elements.  We’ll use n <= 1 for now. Large instance.  Divide into k >= 2 smaller instances.  k = 2, 3, 4, … ?  What does each smaller instance look like?  Sort smaller instances recursively.  How do you combine the sorted smaller instances?

10. Insertion Sort k = 2 First n - 1 elements (a[0:n-2]) define one of the smaller instances; last element (a[n-1]) defines the second smaller instance. a[0:n-2] is sorted recursively. a[n-1] is a small instance.a[0]a[n-1] a[n-2]

11. Insertion Sort Combining is done by inserting a[n-1] into the sorted a[0:n-2]. Complexity is O(n 2 ). Usually implemented nonrecursively.a[0]a[n-1] a[n-2]

12. Selection Sort k = 2 To divide a large instance into two smaller instances, first find the largest element. The largest element defines one of the smaller instances; the remaining n-1 elements define the second smaller instance.a[0]a[n-1] a[n-2]

13. Selection Sort The second smaller instance is sorted recursively. Append the first smaller instance (largest element) to the right end of the sorted smaller instance. Complexity is O(n 2 ). Usually implemented nonrecursively.a[0]a[n-1] a[n-2]

14. Bubble Sort Bubble sort may also be viewed as a k = 2 divide- and-conquer sorting method. Insertion sort, selection sort and bubble sort divide a large instance into one smaller instance of size n - 1 and another one of size 1. All three sort methods take O(n 2 ) time.

15. Divide And Conquer Divide-and-conquer algorithms generally have best complexity when a large instance is divided into smaller instances of approximately the same size. When k = 2 and n = 24, divide into two smaller instances of size 12 each. When k = 2 and n = 25, divide into two smaller instances of size 13 and 12, respectively.

16. Merge Sort k = 2 First ceil(n/2) elements define one of the smaller instances; remaining floor(n/2) elements define the second smaller instance. Each of the two smaller instances is sorted recursively. The sorted smaller instances are combined using a process called merge. Complexity is O(n log n). Usually implemented nonrecursively.

17. Merge Two Sorted Lists A = (2, 5, 6) B = (1, 3, 8, 9, 10) C = () Compare smallest elements of A and B and merge smaller into C. A = (2, 5, 6) B = (3, 8, 9, 10) C = (1)

18. Merge Two Sorted Lists A = (5, 6) B = (3, 8, 9, 10) C = (1, 2) A = (5, 6) B = (8, 9, 10) C = (1, 2, 3) A = (6) B = (8, 9, 10) C = (1, 2, 3, 5)

19. Merge Two Sorted Lists A = () B = (8, 9, 10) C = (1, 2, 3, 5, 6) When one of A and B becomes empty, append the other list to C. O(1) time needed to move an element into C. Total time is O(n + m), where n and m are, respectively, the number of elements initially in A and B.

20. Merge Sort [8, 3, 13, 6, 2, 14, 5, 9, 10, 1, 7, 12, 4] [8, 3, 13, 6, 2, 14, 5][9, 10, 1, 7, 12, 4] [8, 3, 13, 6][2, 14, 5] [8, 3][13, 6] [8][3][13][6] [2, 14][5] [2][14] [9, 10, 1][7, 12, 4] [9, 10][1] [9][10] [7, 12][4] [7][12]

21. Merge Sort [3, 8][6, 13] [3, 6, 8, 13] [8][3][13][6] [2, 14] [2, 5, 14] [2, 3, 5, 6, 8, 13, 14] [5] [2][14] [9, 10] [1, 9, 10] [1] [9][10] [7, 12] [4, 7, 12] [1, 4, 7, 9, 10,12] [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13,14] [4] [7][12]

22. Time Complexity Let t(n) be the time required to sort n elements. t(0) = t(1) = c, where c is a constant. When n > 1, t(n) = t(ceil(n/2)) + t(floor(n/2)) + dn, where d is a constant. To solve the recurrence, assume n is a power of 2 and use repeated substitution. t(n) = O(n log n).

23. Merge Sort Downward pass over the recursion tree.  Divide large instances into small ones. Upward pass over the recursion tree.  Merge pairs of sorted lists. Number of leaf nodes is n. Number of nonleaf nodes is n-1.

24. Time Complexity Downward pass.  O(1) time at each node.  O(n) total time at all nodes. Upward pass.  O(n) time merging at each level that has a nonleaf node.  Number of levels is O(log n).  Total time is O(n log n).

25. Nonrecursive Version Eliminate downward pass. Start with sorted lists of size 1 and do pairwise merging of these sorted lists as in the upward pass.

26. Nonrecursive Merge Sort [8][3][13][6][2][14][5][9][10][1][7][12][4] [3, 8][6, 13][2, 14] [5, 9] [1, 10][7, 12] [4] [3, 6, 8, 13][2, 5, 9, 14][1, 7, 10, 12] [4] [2, 3, 5, 6, 8, 9, 13, 14][1, 4, 7, 10, 12] [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14]

27. Complexity Sorted segment size is 1, 2, 4, 8, … Number of merge passes is ceil(log 2 n). Each merge pass takes O(n) time. Total time is O(n log n). Need O(n) additional space for the merge. Merge sort is slower than insertion sort when n <= 15 (approximately). So define a small instance to be an instance with n <= 15. Sort small instances using insertion sort. Start with segment size = 15.

28. Natural Merge Sort Initial sorted segments are the naturally ocurring sorted segments in the input. Input = [8, 9, 10, 2, 5, 7, 9, 11, 13, 15, 6, 12, 14]. Initial segments are: [8, 9, 10] [2, 5, 7, 9, 11, 13, 15] [6, 12, 14] 2 (instead of 4) merge passes suffice. Segment boundaries have a[i] > a[i+1].

29. Quick Sort Small instance has n <= 1. Every small instance is a sorted instance. To sort a large instance, select a pivot element from out of the n elements. Partition the n elements into 3 groups left, middle and right. The middle group contains only the pivot element. All elements in the left group are <= pivot. All elements in the right group are >= pivot. Sort left and right groups recursively. Answer is sorted left group, followed by middle group followed by sorted right group.

30. Example 6285111041973 Use 6 as the pivot. 2851110419736 Sort left and right groups recursively.

31. Choice Of Pivot Pivot is leftmost element in list that is to be sorted.  When sorting a[6:20], use a[6] as the pivot. Randomly select one of the elements to be sorted as the pivot.  When sorting a[6:20], generate a random number r in the range [6, 20]. Use a[r] as the pivot.

32. Choice Of Pivot Median-of-Three rule. From the leftmost, middle, and rightmost elements of the list to be sorted, select the one with median key as the pivot.  When sorting a[6:20], examine a[6], a[13] ((6+20)/2), and a[20]. Select the element with median (i.e., middle) key.  If a[6].key = 30, a[13].key = 2, and a[20].key = 10, a[20] becomes the pivot.  If a[6].key = 3, a[13].key = 2, and a[20].key = 10, a[6] becomes the pivot.

33. Choice Of Pivot  If a[6].key = 30, a[13].key = 25, and a[20].key = 10, a[13] becomes the pivot. When the pivot is picked at random or when the median-of-three rule is used, we can use the quick sort code of the text provided we first swap the leftmost element and the chosen pivot. pivot swap

34. Partitioning Into Three Groups Sort a = [6, 2, 8, 5, 11, 10, 4, 1, 9, 7, 3]. Leftmost element (6) is the pivot. When another array b is available:  Scan a from left to right (omit the pivot in this scan), placing elements <= pivot at the left end of b and the remaining elements at the right end of b.  The pivot is placed at the remaining position of the b.

35. Partitioning Example Using Additional Array 6285111041973 a b 2851110419736 Sort left and right groups recursively.

36. In-place Partitioning Find leftmost element (bigElement) > pivot. Find rightmost element (smallElement) < pivot. Swap bigElement and smallElement provided bigElement is to the left of smallElement. Repeat.

37. In-Place Partitioning Example 6285111041973 a 683 6235111041978 a 6111 6235110411978 a 6104 623514 11978 a 6104 bigElement is not to left of smallElement, terminate process. Swap pivot and smallElement. 42351411978 a 6106

38. Complexity O(n) time to partition an array of n elements. Let t(n) be the time needed to sort n elements. t(0) = t(1) = c, where c is a constant. When t > 1, t(n) = t(|left|) + t(|right|) + dn, where d is a constant. t(n) is maximum when either |left| = 0 or |right| = 0 following each partitioning.

39. Complexity This happens, for example, when the pivot is always the smallest element. For the worst-case time, t(n) = t(n-1) + dn, n > 1 Use repeated substitution to get t(n) = O(n 2 ). The best case arises when |left| and |right| are equal (or differ by 1) following each partitioning. For the best case, the recurrence is the same as for merge sort.

40. Complexity Of Quick Sort So the best-case complexity is O(n log n). Average complexity is also O(n log n). To help get partitions with almost equal size, change in-place swap rule to:  Find leftmost element (bigElement) >= pivot.  Find rightmost element (smallElement) <= pivot.  Swap bigElement and smallElement provided bigElement is to the left of smallElement. O(n) space is needed for the recursion stack. May be reduced to O(log n) (see Exercise 19.22).

41. Complexity Of Quick Sort To improve performance, define a small instance to be one with n <= 15 (say) and sort small instances using insertion sort.

42. Rank Rank of an element is its position in ascending key order. [2,6,7,8,10,15,18,20,25,30,35,40] rank(2) = 0 rank(15) = 5 rank(20) = 7

43. Selection Problem Given n unsorted elements, determine the k’th smallest element. That is, determine the element whose rank is k-1. Applications  Median score on a test. k = ceil(n/2).  Median salary of Computer Scientists.  Identify people whose salary is in the bottom 10%. First find salary at the 10% rank.

44. Selection By Sorting Sort the n elements. Pick up the element with desired rank. O(n log n) time.

45. Divide-And-Conquer Selection Small instance has n <= 1. Selection is easy. When n > 1, select a pivot element from out of the n elements. Partition the n elements into 3 groups left, middle and right as is done in quick sort. The rank of the pivot is the location of the pivot following the partitioning. If k-1 = rank(pivot), pivot is the desired element. If k-1 < rank(pivot), determine the k’th smallest element in left. If k-1 > rank(pivot), determine the (k-rank(pivot)-1)’th smallest element in right.

46. D&C Selection Example Use 3 as the pivot and partition. rank(pivot) = 5. So pivot is the 6’th smallest element. Find kth element of: 3280111012971 a 12102411978310 a

47. D&C Selection Example If k = 6 (k-1 = rank(pivot)), pivot is the element we seek. If k < 6 (k-1 < rank(pivot)), find k’th smallest element in left partition. If k > 6 (k-1 > rank(pivot)), find (k- rank(pivot)-1)’th smallest element in right partition. 12102411978310 a

48. Time Complexity Worst case arises when the partition to be searched always has all but the pivot.  O(n 2 ) Expected performance is O(n). Worst case becomes O(n) when the pivot is chosen carefully.  Partition into n/9 groups with 9 elements each (last group may have a few more)  Find the median element in each group.  pivot is the median of the group medians.  This median is found using select recursively.

49. Tiling A Defective Chessboard

50. Our Definition Of A Chessboard A chessboard is an n x n grid, where n is a power of 2. 1x1 2x2 4x4 8x8

51. A defective chessboard is a chessboard that has one unavailable (defective) position. 1x1 2x2 4x4 8x8

52. A Triomino A triomino is an L shaped object that can cover three squares of a chessboard. A triomino has four orientations.

53. Tiling A Defective Chessboard Place (n 2 - 1)/3 triominoes on an n x n defective chessboard so that all n 2 - 1 nondefective positions are covered. 1x1 2x2 4x4 8x8

54. Tiling A Defective Chessboard Divide into four smaller chessboards. 4 x 4 One of these is a defective 4 x 4 chessboard.

55. Tiling A Defective Chessboard Make the other three 4 x 4 chessboards defective by placing a triomino at their common corner. Recursively tile the four defective 4 x 4 chessboards.

56. Complexity Let n = 2 k. Let t(k) be the time taken to tile a 2 k x 2 k defective chessboard. t(0) = d, where d is a constant. t(k) = 4t(k-1) + c, when k > 0. Here c is a constant. Recurrence equation for t().

57. Substitution Method t(k) = 4t(k-1) + c = 4[4t(k-2) + c] + c = 4 2 t(k-2) + 4c + c = 4 2 [4t(k-3) + c] + 4c + c = 4 3 t(k-3) + 4 2 c + 4c + c = … = 4 k t(0) + 4 k-1 c + 4 k-2 c +... + 4 2 c + 4c + c = 4 k d + 4 k-1 c + 4 k-2 c +... + 4 2 c + 4c + c = Theta(4 k ) = Theta(number of triominoes placed)