1. Quick Sort Cmput 115 - Lecture 13 Department of Computing Science University of Alberta ©Duane Szafron 2000 Some code in this lecture is based on code from the book: Java Structures by Duane A. Bailey or the companion structure package Revised 1/26/00

2. ©Duane Szafron 1999 2 About This Lecture In this lecture we will learn about a sorting algorithm called the Quick Sort. We will study its implementation and its time and space complexity.

3. ©Duane Szafron 1999 3Outline The Quick Sort Algorithm Quick Sort - Arrays Time and Space Complexity of Quick Sort

4. ©Duane Szafron 1999 4 The Sort Problem Given a collection, with elements that can be compared, put the elements in increasing or decreasing order. 603010204090708050 012345678 102030405060708090 012345678

5. ©Duane Szafron 1999 5 Quick Sort Algorithm 0 Our initial goal is to move one element, called the pivot, to its correct final position so that all elements to the left of it are smaller than it and all elements to the right of it are larger than it. We will call this operation partition(). We select the left element as the pivot. 603010204090708050 012345678 rlp

6. ©Duane Szafron 1999 6 Quick Sort Algorithm 1 Find the rightmost element that is smaller than the pivot element. Exchange the elements and increment the left. 603010204090708050 012345678 lprr 3010204090708060 012345678 rlp

7. ©Duane Szafron 1999 7 Quick Sort Algorithm 2 Find the leftmost element that is larger than the pivot element. Exchange the elements and decrement the right. 503010204060708090 012345678 lpr 503010204090708060 012345678 rlpl

8. ©Duane Szafron 1999 8 Quick Sort Algorithm 3 Find the rightmost element that is smaller than the pivot element. Since the right passes the left, there is no element and the pivot is the final location. 503010204060708090 012345678 rlpr

9. ©Duane Szafron 1999 9 Quick Sort Algorithm 4 To complete the sort, recursively partition the sub-list to the left of the pivot and the sub-list to the right of the pivot.

10. ©Duane Szafron 1999 10 QuickSort Algorithm - Arrays public static void quickSort(Comparable anArray[ ], int size) { // pre: 0 <= size <= anArray.length // post: values in anArray[0..size - 1] are in // ascending order quickSortR(anArray, 0, size - 1); }

11. ©Duane Szafron 1999 11 QuickSortR Algorithm - Arrays private static void quickSortR(Comparable anArray[], int left, int right) { // pre: left <= right // post: anArray[left..right] are ascending int pivot; if (left >= right) return; pivot = partition(anArray, left, right); quickSortR(anArray, left, pivot - 1); quickSortR(anArray, pivot + 1, right); }

12. ©Duane Szafron 1999 12 Partition Algorithm - Arrays private static int partition(Comparable anArray[], int left, int right) { // pre: left <= right // post: data[left] is in the correct sort location and its // index is returned. while (true) { while ((left < right) && (anArray[left].compareTo(anArray[right]) < 0)) right--; //find rightmost element less than left if (left < right) swap(anArray, left++, right); else return left; //pivot is now in final location while ((left < right) && (anArray[left].compareTo(anArray[right]) < 0)) left++; //find leftmost element greater than right if (left < right) swap(anArray, left, right--); else return right; //pivot is now in final location } code based on Bailey pg. 89

13. ©Duane Szafron 1999 13 QuickSort Calling Sequence qSR(a, 0, 8) –p(a,0,8) -> 5 –qSR(a, 0,4) p(a,0,4) -> 4 qSR(a,0,3) –p(a,0,3) -> 3 –qSR(a,0,2) p(a,0,2) -> 1 qSR(a, 0, 0) qSR(a, 2, 2) –qSR(a,4,3) qSR(a,4,3) –qSR(a, 6, 8) p(a, 6, 8) -> 6 qSR(a, 6, 5) qSR(a, 7, 8) –p(a,7,8) -> 7 –qSR(a, 7,6) –qSR(a, 8, 8)

14. ©Duane Szafron 1999 14 Quick Sort Trace 1 qSR(0,8) 012345678 603010204090708050 012345678 603010204090708050 012345678 3010204090708060 p(0,8) 012345678 603010204090708050 012345678 3010204090708060 012345678 503010204060708090 012345678 503010204060708090 012345678 503010204060708090 qSR(0,4) 012345678 503010204060708090

15. ©Duane Szafron 1999 15 Quick Sort Trace 2 qSR(0,4) 012345678 503010204060708090 p(0,4) 012345678 503010204060708090 012345678 503010204060708090 012345678 403010205060708090 012345678 403010205060708090 012345678 403010205060708090 qSR(0,3) 012345678 403010205060708090 p(0,3) 012345678 403010205060708090

16. ©Duane Szafron 1999 16 Quick Sort Trace 3 p(0,3) - cont 012345678 403010205060708090 012345678 403010205060708090 012345678 203010405060708090 012345678 203010405060708090 012345678 203010405060708090 qSR(0,2) 012345678 203010405060708090 p(0,2) 012345678 203010405060708090 012345678 203010405060708090 012345678 103020405060708090 012345678 103020405060708090

17. ©Duane Szafron 1999 17 Quick Sort Trace 4 012345678 103020405060708090 012345678 102030405060708090 012345678 102030405060708090 qSR(0,0) 012345678 102030405060708090 qSR(2,2) 012345678 102030405060708090 qSR(4,3) 012345678 102030405060708090 qSR(6,8) 012345678 102030405060708090

18. ©Duane Szafron 1999 18 Quick Sort Trace 5 qSR(7,8) 012345678 102030405060708090 qSR(6,8) 012345678 102030405060708090 p(6,8) 012345678 102030405060708090 012345678 102030405060708090 012345678 102030405060708090 qSR(6,5) 012345678 102030405060708090 p(7,8) 012345678 102030405060708090 012345678 102030405060708090

19. ©Duane Szafron 1999 19 Quick Sort Trace 6 012345678 102030405060708090 012345678 102030405060708090 qSR(7,6) 012345678 102030405060708090 qSR(8,8) 012345678 102030405060708090

20. ©Duane Szafron 1999 20 Counting Comparisons How many comparison operations are required for a quick sort of an n-element collection? The recursive sort method calls partition() once, and then divides the collection. if (left >= right) return; pivot = partition(anArray, left, right); quickSortR(anArray, left, pivot - 1); quickSortR(anArray, pivot + 1, right); Every call to partition() for a list of size k > 1, performs k-1 comparisons.

21. ©Duane Szafron 1999 21 Comparisons - Best Case 1 In the best case, the pivot is always in the middle of the list being sorted. Count comparisons, C(k), for k = 15: C(15) = 14 + C(7) + C(7) = 14 + (6+2+2) + (6+2+2) = 34 In general, we see that C(k) = k-1 + 2*C((k-1)/2) for k = 2 m -1 k=7 k=3 k=1 k=3 k=1 k=7 k=3 k=1 k=3 k=1 6 14 6 2222 best case comparisons per call to partition n=k=15

22. ©Duane Szafron 1999 22 Comparisons - Best Case 2 To find a general formula for C(n), we need to solve the recurrence relation: C(n) = 2*C((n-1)/2) for n >= 3 Solving recurrence relations is beyond the scope of this course and is considered in C204. The solution of this recurrence relation is: C(n) = (n+1) log(n+1) - 2n for n >= 3 Therefore, in the best case, the total number of comparisons is: (n+1)*log(n+1) - 2n = O(n log(n))

23. ©Duane Szafron 1999 23 Check the Recurrence Relation Check that the solution is valid: C(n) = (n+1) log(n+1) - 2n for n >= 3, Checking: C(3) = 4*log(4) - 6 = 4*2 - 6 = 8 - 6 = 2 Checking: C(7) = 8*log(8) - 14 = 8*3 - 14 = 24 - 14 = 10 Checking: C(15) = 16*log(16)-30 = 16*4-30 = 64-30 = 34 Checking: C(8) = 9*log(9) - 16 ~ 9*3.17 - 16 ~ 12.53 ~ 13 k=3 k=1 k=4 k=1k=2 k=0k=1 2 7 3 1 n=k=8 Number of comparisons in this partitioning

24. ©Duane Szafron 1999 24 Prove Recurrence Solution is Valid Prove that a solution to: C(n) = n-1 + 2*C((n-1)/2) is: C(n) = (n+1) log(n+1) - 2n for n >= 3 Note that: C((n-1)/2) = [(n-1)/2 + 1]*log [(n-1)/2 + 1] -2[(n-1)/2] = ((n+1)/2) *log ((n+1)/2) - (n-1) Therefore: 2*C((n-1)/2) + (n-1) = 2*{((n+1)/2)*log ((n+1)/2) - (n-1)} + (n-1) = (n+1)*log ((n+1)/2) - 2*(n-1) + (n-1) = (n+1)*{log (n+1) - log (2)} - 2n + 2 + n -1 = (n+1)*log (n+1) - (n+1)*log (2) - n + 1 = (n+1)*log (n+1) - (n+1)*1 - n + 1 = (n+1)*log (n+1) - n + 1 - n + 1 = (n+1)*log (n+1) - 2n = C(n)

25. ©Duane Szafron 1999 25 Comparisons - Worst Case In the worst case, the pivot is at one end of the list so a list of size k gets divided into a list of size 0 and a list of size k-1. In this case there are n calls to partition(). Since there are k-1 comparisons in partition() for a list of size k, we have a total number of comparisons: (n-1) + (n-2) + … + 1 = (n-1)*n/2 = O(n 2 ) The bad news is that the worst case occurs when the list is sorted (or near sorted). Why? Think about the consequences of this!

26. ©Duane Szafron 1999 26 Comparisons - Average Case The average case must be between the best case and the worst case, but since the best case is O(n log(n)) and the worst case is so the average case is O(n 2 ), some analysis is necessary to find the answer. Analysis yields a complex recurrence relation. The average case number of comparisons is approximately: 1.386*n*log(n) - 2.846*n Therefore, the average case time complexity is: O(n log(n)).

27. ©Duane Szafron 1999 27 Counting Assignments How many assignment operations are required for a quick sort of an n-element collection? In general there are far fewer assignments than comparisons, since there are 3 assignments per swap, and swap is not called for every comparison: while ((left < right) &&(anArray[left].compareTo(anArray[right]) < 0)) right--; if (left < right) swap(anArray, left++, right); else return left; Therefore we ignore the assignments.

28. ©Duane Szafron 1999 28 Time Complexity of Quick Sort Best case O(n log(n)) for comparisons. Worst case O(n 2 ) for comparisons. Average case O(n log(n)) for comparisons. Note that the quick sort is inferior to insertion sort and merge sort if the list is sorted, nearly sorted, or reverse sorted.

29. ©Duane Szafron 1999 29 Space Complexity of Quick Sort This sort appears to only need one temporary memory element for the swap. However, in the worst case, there are n recursive calls, each of which requires three parameters: a reference to the array, and two ints, left and right. This means that there are 3*n more words of storage necessary. In the average and best cases there are only log(n) recursive calls, so this extra storage is negligible. Note that we ignored this extra space in the merge sort since there were only log(n) recursive calls.