Divide and Conquer Sorting Algorithms CS 105

A new sorting strategy Divide-and-Conquer Given the collection of n elements to sort: perform the sort in three steps Divide step: split the collection S into two subsets, S1 and S2 Recur step: sort S1 and S2 separately Conquer step: combine the two lists into one sorted list Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 2

Quick sort and Merge sort Two algorithms adopt this divide-and-conquer strategy Quick sort Work is carried out in the divide step using a pivot element Conquer step is trivial Mere sort Divide step is trivial – just split the list into two equal parts Work is carried out in the conquer step by merging two sorted lists Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 3

Quick sort: divide step In the divide step, select a pivot from the array (say, the last element) Split the list/array S using the pivot: S1 consists of all elements < pivot, S2 consists of all elements > pivot 85 24 63 45 17 31 96 50 S1 S2 24 45 17 31 85 63 96 50 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 4

Quick sort: conquer step After sorting S1 and S2, combine the sorted lists so that S1 is on the left, the pivot is in the middle, and S2 is on the right S1-sorted S2-sorted 17 24 31 45 63 85 96 50 17 24 31 45 50 63 85 96 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 5

Quick sort with recur step 85 24 63 45 17 31 96 50 Divide S1 S2 24 45 17 31 85 63 96 50 Recur S1-sorted S2-sorted 17 24 31 45 63 85 96 50 Conquer 17 24 31 45 50 63 85 96 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 6

Implementing quick sort It is preferable if we can perform quick-sort in-place; i.e., we sort by swapping elements in the array, perhaps using some temporary variables Plan: algorithm QSort( S, a, b ) sorts the sublist in the array S from index a to index b QSort( L, 0, n-1 ) will sort an array L of length n Within the QSort( S, a, b ) algorithm, there will be recursive calls to QSort on smaller ranges within the range a…b. Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 7

base case: a…b range contains 0 or 1 element Algorithm QSort Algorithm QSort( S, a, b ) if ( a < b ) p  S[b] rearrange S so that: S[a]…S[x-1] are elements < p S[x] = p S[x+1]…S[b] are elements > p QSort( S, a, x-1 ) QSort( S, x+1, b ) base case: a…b range contains 0 or 1 element Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 8

Rearranging a sublist in S p  S[b], l  a, r  b - 1 while l <= r do // find an element larger than the pivot while l <= r and S[l] <= p do l  l + 1 // find an element smaller than the pivot while r >= l and S[r] >= p do r  r – 1 if l < r then swap ( S[l], S[r] ) // swap the two elements swap( S[l], S[b] ) // place pivot in proper place Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 9

Time complexity of quick sort First, note that rearranging a sublist takes O( n ) time, where n is the length of the sublist Requires scanning the list from both ends until both l and r pointers meet O( n ) even if loops are nested within a loop Rearranging sublists is all that the quick sort algorithm does Need to find out how often the sort would perform the rearrange operation Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 10

Time complexity of quick sort Suppose the pivots always split the lists into two lists of roughly equal size 1 list of length n 2 lists of length n/2 4 lists of length n/4 n lists of length 1 ... Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 11

Time complexity of quick sort Each level takes O( n ) time for sublist rearranging Assuming an even split caused by each pivot, there will be around log n levels Therefore, quick sort takes O( n log n ) time But… Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 12

Time complexity of quick sort In the worst case, the pivot might split the lists such that there is only 1 element in one partition (not an even split) There will be n levels Each level requires O( n ) time for sublist rearranging Quick sort takes O( n2 ) time in the worst case Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 13

Merge sort Another sorting algorithm using the divide-and-conquer paradigm This time, the hard work is carried out in the conquer phase instead of the divide phase Divide: split the list S[0..n-1] by taking the middle index m ( = (0 + n-1) / 2 ) Recur: recursively sort S[0..m] and S[m+1..n-1] Conquer: merge the two sorted lists (how?) Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 14

Merge sort 85 24 63 45 17 31 96 50 Divide S1 S2 85 24 63 45 17 31 96 50 Recur S1-sorted S2-sorted 24 45 63 85 17 31 50 96 Conquer 17 24 31 45 50 63 85 96 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 15

Merging two sorted lists Requires inspecting the “head elements” of both lists Whichever element is smaller, that elements goes first, and the head element for the “winning” list is updated so that it refers to the next element in that list Repeat the process until all the elements in both lists have been processed Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 16

Merge sort time complexity Divide step ensures that the sublist split is done evenly O( log n ) levels Conquer/merge step takes O( n ) time per level Time complexity is O( n log n ), guaranteed Disadvantage: hard to carry out the merge step in-place; temporary array/list is necessary if we want a simple implementation Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 17

Time complexity summary Algorithm Best case Worst Heap sort O( n log n ) Quick sort O(n2 ) Merge sort Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 18

Summary and final points Quick sort has a bad worst case but works very well in practice; O( n log n ) on the average Merge sort is difficult to implement in-place but O( n log n ) complexity is guaranteed But it doesn’t perform well in practice Heap sort is a guaranteed O( n log n ) algorithm that is not that difficult to implement Reasonable alternative to quick sort Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved HeapSort Slide 19