Quick Sort Cmput Lecture 13 Department of Computing Science University of Alberta ©Duane Szafron 2000 Some code in this lecture is based on code from the book: Java Structures by Duane A. Bailey or the companion structure package Revised 1/26/00

©Duane Szafron About This Lecture In this lecture we will learn about a sorting algorithm called the Quick Sort. We will study its implementation and its time and space complexity.

©Duane Szafron Outline The Quick Sort Algorithm Quick Sort - Arrays Time and Space Complexity of Quick Sort

©Duane Szafron The Sort Problem Given a collection, with elements that can be compared, put the elements in increasing or decreasing order

©Duane Szafron Quick Sort Algorithm 0 Our initial goal is to move one element, called the pivot, to its correct final position so that all elements to the left of it are smaller than it and all elements to the right of it are larger than it. We will call this operation partition(). We select the left element as the pivot rlp

©Duane Szafron Quick Sort Algorithm 1 Find the rightmost element that is smaller than the pivot element. Exchange the elements and increment the left lprr rlp

©Duane Szafron Quick Sort Algorithm 2 Find the leftmost element that is larger than the pivot element. Exchange the elements and decrement the right lpr rlpl

©Duane Szafron Quick Sort Algorithm 3 Find the rightmost element that is smaller than the pivot element. Since the right passes the left, there is no element and the pivot is the final location rlpr

©Duane Szafron Quick Sort Algorithm 4 To complete the sort, recursively partition the sub-list to the left of the pivot and the sub-list to the right of the pivot.

©Duane Szafron QuickSort Algorithm - Arrays public static void quickSort(Comparable anArray[ ], int size) { // pre: 0 <= size <= anArray.length // post: values in anArray[0..size - 1] are in // ascending order quickSortR(anArray, 0, size - 1); }

©Duane Szafron QuickSortR Algorithm - Arrays private static void quickSortR(Comparable anArray[], int left, int right) { // pre: left <= right // post: anArray[left..right] are ascending int pivot; if (left >= right) return; pivot = partition(anArray, left, right); quickSortR(anArray, left, pivot - 1); quickSortR(anArray, pivot + 1, right); }

©Duane Szafron Partition Algorithm - Arrays private static int partition(Comparable anArray[], int left, int right) { // pre: left <= right // post: data[left] is in the correct sort location and its // index is returned. while (true) { while ((left < right) && (anArray[left].compareTo(anArray[right]) < 0)) right--; //find rightmost element less than left if (left < right) swap(anArray, left++, right); else return left; //pivot is now in final location while ((left < right) && (anArray[left].compareTo(anArray[right]) < 0)) left++; //find leftmost element greater than right if (left < right) swap(anArray, left, right--); else return right; //pivot is now in final location } code based on Bailey pg. 89

©Duane Szafron QuickSort Calling Sequence qSR(a, 0, 8) –p(a,0,8) -> 5 –qSR(a, 0,4) p(a,0,4) -> 4 qSR(a,0,3) –p(a,0,3) -> 3 –qSR(a,0,2) p(a,0,2) -> 1 qSR(a, 0, 0) qSR(a, 2, 2) –qSR(a,4,3) qSR(a,4,3) –qSR(a, 6, 8) p(a, 6, 8) -> 6 qSR(a, 6, 5) qSR(a, 7, 8) –p(a,7,8) -> 7 –qSR(a, 7,6) –qSR(a, 8, 8)

©Duane Szafron Quick Sort Trace 1 qSR(0,8) p(0,8) qSR(0,4)

©Duane Szafron Quick Sort Trace 2 qSR(0,4) p(0,4) qSR(0,3) p(0,3)

©Duane Szafron Quick Sort Trace 3 p(0,3) - cont qSR(0,2) p(0,2)

©Duane Szafron Quick Sort Trace qSR(0,0) qSR(2,2) qSR(4,3) qSR(6,8)

©Duane Szafron Quick Sort Trace 5 qSR(7,8) qSR(6,8) p(6,8) qSR(6,5) p(7,8)

©Duane Szafron Quick Sort Trace qSR(7,6) qSR(8,8)

©Duane Szafron Counting Comparisons How many comparison operations are required for a quick sort of an n-element collection? The recursive sort method calls partition() once, and then divides the collection. if (left >= right) return; pivot = partition(anArray, left, right); quickSortR(anArray, left, pivot - 1); quickSortR(anArray, pivot + 1, right); Every call to partition() for a list of size k > 1, performs k-1 comparisons.

©Duane Szafron Comparisons - Best Case 1 In the best case, the pivot is always in the middle of the list being sorted. Count comparisons, C(k), for k = 15: C(15) = 14 + C(7) + C(7) = 14 + (6+2+2) + (6+2+2) = 34 In general, we see that C(k) = k-1 + 2*C((k-1)/2) for k = 2 m -1 k=7 k=3 k=1 k=3 k=1 k=7 k=3 k=1 k=3 k= best case comparisons per call to partition n=k=15

©Duane Szafron Comparisons - Best Case 2 To find a general formula for C(n), we need to solve the recurrence relation: C(n) = 2*C((n-1)/2) for n >= 3 Solving recurrence relations is beyond the scope of this course and is considered in C204. The solution of this recurrence relation is: C(n) = (n+1) log(n+1) - 2n for n >= 3 Therefore, in the best case, the total number of comparisons is: (n+1)*log(n+1) - 2n = O(n log(n))

©Duane Szafron Check the Recurrence Relation Check that the solution is valid: C(n) = (n+1) log(n+1) - 2n for n >= 3, Checking: C(3) = 4*log(4) - 6 = 4*2 - 6 = = 2 Checking: C(7) = 8*log(8) - 14 = 8* = = 10 Checking: C(15) = 16*log(16)-30 = 16*4-30 = = 34 Checking: C(8) = 9*log(9) - 16 ~ 9* ~ ~ 13 k=3 k=1 k=4 k=1k=2 k=0k= n=k=8 Number of comparisons in this partitioning

©Duane Szafron Prove Recurrence Solution is Valid Prove that a solution to: C(n) = n-1 + 2*C((n-1)/2) is: C(n) = (n+1) log(n+1) - 2n for n >= 3 Note that: C((n-1)/2) = [(n-1)/2 + 1]*log [(n-1)/2 + 1] -2[(n-1)/2] = ((n+1)/2) *log ((n+1)/2) - (n-1) Therefore: 2*C((n-1)/2) + (n-1) = 2*{((n+1)/2)*log ((n+1)/2) - (n-1)} + (n-1) = (n+1)*log ((n+1)/2) - 2*(n-1) + (n-1) = (n+1)*{log (n+1) - log (2)} - 2n n -1 = (n+1)*log (n+1) - (n+1)*log (2) - n + 1 = (n+1)*log (n+1) - (n+1)*1 - n + 1 = (n+1)*log (n+1) - n n + 1 = (n+1)*log (n+1) - 2n = C(n)

©Duane Szafron Comparisons - Worst Case In the worst case, the pivot is at one end of the list so a list of size k gets divided into a list of size 0 and a list of size k-1. In this case there are n calls to partition(). Since there are k-1 comparisons in partition() for a list of size k, we have a total number of comparisons: (n-1) + (n-2) + … + 1 = (n-1)*n/2 = O(n 2 ) The bad news is that the worst case occurs when the list is sorted (or near sorted). Why? Think about the consequences of this!

©Duane Szafron Comparisons - Average Case The average case must be between the best case and the worst case, but since the best case is O(n log(n)) and the worst case is so the average case is O(n 2 ), some analysis is necessary to find the answer. Analysis yields a complex recurrence relation. The average case number of comparisons is approximately: 1.386*n*log(n) *n Therefore, the average case time complexity is: O(n log(n)).

©Duane Szafron Counting Assignments How many assignment operations are required for a quick sort of an n-element collection? In general there are far fewer assignments than comparisons, since there are 3 assignments per swap, and swap is not called for every comparison: while ((left < right) &&(anArray[left].compareTo(anArray[right]) < 0)) right--; if (left < right) swap(anArray, left++, right); else return left; Therefore we ignore the assignments.

©Duane Szafron Time Complexity of Quick Sort Best case O(n log(n)) for comparisons. Worst case O(n 2 ) for comparisons. Average case O(n log(n)) for comparisons. Note that the quick sort is inferior to insertion sort and merge sort if the list is sorted, nearly sorted, or reverse sorted.

©Duane Szafron Space Complexity of Quick Sort This sort appears to only need one temporary memory element for the swap. However, in the worst case, there are n recursive calls, each of which requires three parameters: a reference to the array, and two ints, left and right. This means that there are 3*n more words of storage necessary. In the average and best cases there are only log(n) recursive calls, so this extra storage is negligible. Note that we ignored this extra space in the merge sort since there were only log(n) recursive calls.