1. Divide and Conquer Sorting Algorithms
CS 105

2. A new sorting strategy Divide-and-Conquer
Given the collection of n elements to sort: perform the sort in three steps
Divide step: split the collection S into two subsets, S1 and S2
Recur step: sort S1 and S2 separately
Conquer step: combine the two lists into one sorted list
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 2

3. Quick sort and Merge sort
Two algorithms adopt this divide-and-conquer strategy
Quick sort
Work is carried out in the divide step using a pivot element
Conquer step is trivial
Mere sort
Divide step is trivial – just split the list into two equal parts
Work is carried out in the conquer step by merging two sorted lists
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 3

4. Quick sort: divide step
In the divide step, select a pivot from the array (say, the last element)
Split the list/array S using the pivot: S1 consists of all elements < pivot, S2 consists of all elements > pivot 85 24 63 45 17 31 96 50 S1 S2 24 45 17 31 85 63 96 50
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 4

5. Quick sort: conquer step
After sorting S1 and S2, combine the sorted lists so that S1 is on the left, the pivot is in the middle, and S2 is on the right
S1-sorted
S2-sorted 17 24 31 45 63 85 96 50 17 24 31 45 50 63 85 96
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 5

6. Quick sort with recur step85 24 63 45 17 31 96 50
Divide S1 S2 24 45 17 31 85 63 96 50
Recur
S1-sorted
S2-sorted 17 24 31 45 63 85 96 50
Conquer 17 24 31 45 50 63 85 96
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 6

7. Implementing quick sort
It is preferable if we can perform quick-sort in-place; i.e., we sort by swapping elements in the array, perhaps using some temporary variables
Plan: algorithm QSort( S, a, b ) sorts the sublist in the array S from index a to index b
QSort( L, 0, n-1 ) will sort an array L of length n
Within the QSort( S, a, b ) algorithm, there will be recursive calls to QSort on smaller ranges within the range a…b.
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 7

8. base case: a…b range contains 0 or 1 element
Algorithm QSort
Algorithm QSort( S, a, b )
if ( a < b )
p  S[b]
rearrange S so that:
S[a]…S[x-1] are elements < p
S[x] = p
S[x+1]…S[b] are elements > p
QSort( S, a, x-1 )
QSort( S, x+1, b )
base case: a…b range contains 0 or 1 element
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 8

9. Rearranging a sublist in S
p  S[b], l  a, r  b - 1
while l <= r do
// find an element larger than the pivot
while l <= r and S[l] <= p do
l  l + 1
// find an element smaller than the pivot
while r >= l and S[r] >= p do
r  r – 1
if l < r then
swap ( S[l], S[r] ) // swap the two elements
swap( S[l], S[b] ) // place pivot in proper place
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 9

10. Time complexity of quick sort
First, note that rearranging a sublist takes O( n ) time, where n is the length of the sublist
Requires scanning the list from both ends until both l and r pointers meet
O( n ) even if loops are nested within a loop
Rearranging sublists is all that the quick sort algorithm does
Need to find out how often the sort would perform the rearrange operation
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 10

11. Time complexity of quick sort
Suppose the pivots always split the lists into two lists of roughly equal size
1 list of length n
2 lists of length n/2
4 lists of length n/4
n lists of length 1
...
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 11

12. Time complexity of quick sort
Each level takes O( n ) time for sublist rearranging
Assuming an even split caused by each pivot, there will be around log n levels
Therefore, quick sort takes O( n log n ) time
But…
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 12

13. Time complexity of quick sort
In the worst case, the pivot might split the lists such that there is only 1 element in one partition (not an even split)
There will be n levels
Each level requires O( n ) time for sublist rearranging
Quick sort takes O( n2 ) time in the worst case
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 13

14. Merge sort
Another sorting algorithm using the divide-and-conquer paradigm
This time, the hard work is carried out in the conquer phase instead of the divide phase
Divide: split the list S[0..n-1] by taking the middle index m ( = (0 + n-1) / 2 )
Recur: recursively sort S[0..m] and S[m+1..n-1]
Conquer: merge the two sorted lists (how?)
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 14

15. Merge sort 85 24 63 45 17 31 96 50
Divide S1 S2 85 24 63 45 17 31 96 50
Recur
S1-sorted
S2-sorted 24 45 63 85 17 31 50 96
Conquer 17 24 31 45 50 63 85 96
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 15

16. Merging two sorted lists
Requires inspecting the “head elements” of both lists
Whichever element is smaller, that elements goes first, and the head element for the “winning” list is updated so that it refers to the next element in that list
Repeat the process until all the elements in both lists have been processed
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 16

17. Merge sort time complexity
Divide step ensures that the sublist split is done evenly
O( log n ) levels
Conquer/merge step takes O( n ) time per level
Time complexity is O( n log n ), guaranteed
Disadvantage: hard to carry out the merge step in-place; temporary array/list is necessary if we want a simple implementation
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 17

18. Time complexity summary
Algorithm
Best
case
Worst
Heap sort
O( n log n )
Quick sort
O(n2 )
Merge sort
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 18

19. Summary and final points
Quick sort has a bad worst case but works very well in practice; O( n log n ) on the average
Merge sort is difficult to implement in-place but O( n log n ) complexity is guaranteed
But it doesn’t perform well in practice
Heap sort is a guaranteed O( n log n ) algorithm that is not that difficult to implement
Reasonable alternative to quick sort
Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved
HeapSort
Slide 19