1. Data Structures & Algorithms
QuickSort 1

2. QuickSort One of the top 20 algorithms
Invented by C. A. R. Hoare in 1960
Widely implemented
Widely studied
Desirable features:
Works in-place (small auxiliary stack)
N lg N time on average
Very short inner loop 2

3. QuickSort One of the top 20 algorithms
Invented by C. A. R. Hoare in 1960
Widely implemented
Widely studied
Downside:
Not stable
N2 time worst case
Susceptible to bad implementation 3

4. QuickSort Divide-and-Conquer method Partition array into two parts
Small elements
Large elements
Sort each part independently
Partitioning is the central component:
Element a[i] is in its final place
Element a[j] <= a[i] for all j < i
Element a[j] >= a[i] for all j > i 4

5. QuickSort Divide-and-Conquer method Partition array into two parts
Small elements
Large elements
Sort each part independently
void quickSort(Item a[], int l, int r) {
if (r <= l) return;
int i = partition(a, l, r);
quickSort(a, l, i-1);
quickSort(a, i+1, r); } 5

6. QuickSort Partition procedure is crucial to success.
If the two parts are “about equal” in size, then quickSort is N lg N time and fast.
First cut: pick a[r] to be the pivot element.
void quickSort(Item a[], int l, int r) {
if (r <= l) return;
int i = partition(a, l, r);
quickSort(a, l, i-1);
quickSort(a, i+1, r); } 6

7. QuickSort Partition procedure is crucial to success.
Pick a[r] to be the pivot element.
int partition(Item a[], int l, int r)
{ int i = l-1, j = r; Item v = a[r];
for (;;) {
while (a[++i] < v);
while (a[--j] > v) if (j == l) break;
if (i >= j) break;
exch(a[i], a[j]); }
exch(a[i], a[r]);
return i; 7

8. QuickSort PartitioningL G O R I T H M A L G H I R T O M A L G O R I T H M A L G H I M T O R A L G O R I T H M A L G H I M T O R
return 5 A L G H R I T O M A L G H I M T O R A L G H R I T O M A L G H I M T O R A L G H R I T O M A L G H I M T O R A L G H I R T O M A H G L I M T O R A L G H I R T O M A H G L I M T O R 8

9. QuickSort PartitioningH G L I M T O R A G H I L M T O R A H G I L M T O R A G H I L M T O R A H G I L M T O R
return 3 A G H I L M T O R A H G I L M T O R A G H I L M O T R A H G I L M T O R A G H I L M O T R A H G I L M T O R A G H I L M O R T A G H I L M T O R A G H I L M O R T A G H I L M T O R
return 1 A G H I L M O R T 9

10. QuickSort
Property 7.1: QuickSort uses about N2/2 comparisons in the worst case.
Prf: On a sorted file, QuickSort will make partitions of size N-1 and 0 on a file of length N, as the pivot is already in place. Partitioning takes N comparisons, so the total is N + (N-1) + (N-2) + … , which is (N+1)N/2 10

11. QuickSort
Property 7.2: QuickSort uses about 2 ln N comparisons in the average case.
Prf: On a random file, QuickSort is equally likely to make the left partition of any size k between 0 and N-1, with the right partition of size N-k-1.
The expected cost is then
C(N) = N+1 + (1/N)Sum(C(k)+C(N-k-1)) for N > 1, with C(0) = C(1) = 0. 11

12. QuickSort Property 7.2: QuickSort uses about
2 ln N comparisons in the average case.
Prf: The expected cost is then
C(N) = N+1 + (1/N)Sum(C(k)+C(N-k-1)) = N+1 + (2/N)Sum(C(k))
since each C(k) is counted twice (once on the left, once on the right).
Telescoping, we get
NC(N) = (N+1)C(N-1) + 2N 12

13. QuickSort Property 7.2: QuickSort uses about
2 ln N comparisons in the average case.
Prf: Dividing by N(N+1) we get
C(N)/(N+1) = C(N-1)/N + 2/(N+1)
= C(2)/3 + Sum(2/(k+1)) 2<k<N+1
which is about 2 ln N using an integral approximation. 13

14. QuickSort Stack size can be an issue with any recursive algorithm.
QuickSort has a maximum stack size proportional to lg N for random files,
but for degenerate cases it can be N.
(What are these degenerates?)
Policy of putting the larger partition on the stack ensures that the stack is of minimal size, lg N in the worst case. 14

15. QuickSort
Small subarrays are another issue. This is generally the case with recursive algorithms – they often generate many small cases, so efficiency for small cases affects running time.
For QuickSort, the approach is to use InsertionSort for small subarrays.
How small?
Typical value of “small” is less than 10 or so. 15

16. QuickSort
The most critical issue is partitioning the input array into more or less equal parts.
For QuickSort, the approach is to use Median-of-Three partitioning.
Idea is that the array is “sampled” to find a good pivot – one random sample is probably good, but three is usually much better.
Why not 5? Or 10? Or 1000?
Sampling also costs time, so strike a balance – three turns out pretty well. 16

17. QuickSort Duplicate keys can also be pesky!
QuickSort isn’t bad, it just doesn’t recognize the case where all the elements in a subarray are equal.
Hence, there is room for improvement.
The goal is to partition the array into three parts – less than the pivot, equal to the pivot, and greater than the pivot.
(i.e., just expand the middle part of size 1 that was the pivot to include all the elements equal to the pivot). 17

18. QuickSort
Approach by Bentley and McIlroy (1993) keeps keys equal to the pivot found in the left subarray on the far left, and those found in the right subarray to the far right, until the location for the pivot is found.
Then the equal key elements are swapped within their side to be next to the pivot. 18

19. QuickSort Approach by Bentley and McIlroy (1993) Key same as pivot
Key distinct from pivot
Item in final position 19

20. Medians and Selection A common problem is to find the median.
Can be done easily by sorting…
But this is expensive –
Special case of selection problem (pick kth smallest element in set)
Since we must examine every element (the k-1 that are smaller and the N-k-1 that are larger), not much harder to return all the k smallest elements of a file. 20

21. Medians and Selection Faster ways to do selection:
SelectionSort, but stop after first k sorted
Takes time kN, OK for small k.
Other methods run in time N log k (later…)
QuickSort can do it in linear time on average for all values of k! 21

22. Medians and Selection Faster ways to do selection:
void select(Item a[], int l, int r, int k) {
if (r <= l) return;
int i = partition(a, l, r);
if (i > k) select(a, l, i-1, k);
if (i < k) select(a, i+1, r, k-i-1); } 22

23. Selection
Select k
Select k
Select k-i2
Select k-i2
Select k-i2-i4 23

24. Medians and Selection
Property 7.4: QuickSort based selection is linear time on average.
The full proof is complicated, but roughly a large array should be divided about in half each partitioning, taking time about
N + N/2 + N/4 + … = 2N. 24

25. Summary Covered QuickSort algorithm
analysis (best, worst, average cases)
pivot selection
small arrays
duplicate keys
Selection
problem
quicksort approach to solution 25