1. CSC 213 Lecture 12: Quick Sort & Radix/Bucket Sort

2. Quick-Sort (§ 10.2) Quick-sort is another divide-and-conquer sorting algorithm: Divide: pick an element x (the pivot) and split into 2 partitions:  L -- elements less than x  G -- elements equal to or bigger than x Recur : sort L and G Conquer: merge L, x and G x x L G x

3. Divide Partitioning is slightly more comple than merge-sort : In turn, remove each element y from S Compare y and pivot. Insert y into L or G depending on comparison result Only insert and remove first/last elements, so each insertion and removal takes O(1) time Partitioning takes _______ time Algorithm partition(S, C, pivot) while  S.isEmpty() y  S.removeFirst() comp = C.compare(y, pivot) if comp < 0 L.insertLast(y) else // e.g., comp > 0 G.insertLast(y) return L, pivot, G

4. Execution Tree Can depict execution using binary tree Node still represent recursive quick-sort calls  Left side shows initial sequence and highlights pivot  Right side shows final (sorted) sequence Root node is the initial call Leaves show base cases -- calls of size 0 or 1 7 4 9 6 2  2 4 6 7 9 4 2  2 47 9  7 9 2  29  9

5. 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 Execution Example 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 First, select a pivot

6. Execution Example Partition, recurse and select that pivot 2 4 3 1  2 4 7 9 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 2 4 3 1  2 4 7 9

7. Execution Example Partition, recurse and enjoy the base case 2 4 3 1  2 4 7 1  11  1 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9

8. Execution Example Recursive call to sort the G partition 2 4 3 1  1 2 3 4 1  11  14 3  3 4 9  94  44  4 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 4 3  3 4

9. Execution Example Merge: Add entries from L, pivot, & then G 2 4 3 1  1 2 3 4 1  11  14 3  3 4 9  94  44  4 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9

10. Execution Example 7 9 7 1  1 3 8 6 2 4 3 1  1 2 3 4 1  11  14 3  3 4 9  94  44  4 Recursively solve G partition 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 7 9 7 1  1 3 8 6

11. Execution Example Recursive calls and handle the base case 7 9 7 1  1 3 8 6 8  8 2 4 3 1  1 2 3 4 1  11  14 3  3 4 9  94  44  4 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 7 9  7 9 9  99  99  9

12. Execution Example 8  8 2 4 3 1  1 2 3 4 1  11  14 3  3 4 9  94  44  4 7 2 9 4 3 7 6 1  1 2 3 4 6 7 8 9 7 9 7 1  1 7 7 9 Merge: Add entries from L, pivot, & then G 7 9  7 9 9  99  99  9

13. Execution Example Merge: Add entries from L then E then G 8  8 2 4 3 1  1 2 3 4 1  11  14 3  3 4 9  94  44  4 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 7 9 7 1  1 7 7 9 7 9  7 9 9  99  99  9

14. Worst-case Running Time Worst possible case for quick-sort : we keep pivoting around the minimum or maximum elements Either L or G will have size n  1 and the other has size 0 Big-Oh Running time is therefore equal to the sum n  (n  1)  …  2  Total running time is therefore ____________ depthtime 0n 1 n  1 …… 1 …

15. Quick-Sort Pivot In theory, any Entry can be the pivot Algorithm only requires there be one In reality, want median Entry as pivot Traditionally used first or last Entry This means data already sorted or in reverse order is the worst case  Also fairly common cases. Oops, our Bad. Now usually select an Entry at random

16. Expected Running Time Consider recursive call of quick-sort on a sequence of size n Good call: both L and G have fewer than 3n / 4 entries Bad call: either L or G has more than 3n / 4 entries ½ of the possible pivots result in a good call 7 9 7 1  1 7 2 9 4 3 7 6 1 9 2 4 3 1 7 2 9 4 3 7 61 7 2 9 4 3 7 6 1 Good callBad call Good pivotsBad pivots 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

17. Expected Running Time Toss coin 2 n times to get n heads on average For node at depth i, we can expect i / 2 ancestors were good calls Input sequence would be at most n *(¾) i /2 Expect nodes at depth 2* log 4/3 n to be leaves Expected tree height = O (2*log n ) = O (log n ) Process nearly all of the original entries at each depth Expected running time of quick-sort is ______

18. We Can Rebuild Her… Of previous sorts, only Bubble-sort does not use additional data structures Selection-sort & Insertion-sort can use a Sequence-based priority queue Heap-sort can use a heap-based priority queue (though this can be avoided) Merge-sort uses additional Sequences We can modify Quick-sort to run in- place

19. In-Place Quick-Sort During partitioning, rearrange entries in the sequence such that: entries smaller than pivot (e.g., L ) have rank less than l pivot has rank between l entries larger than pivot (e.g., G ) have rank greater than l First partition S into L and G and then move pivot in between these partitions

20. In-Place Partitioning 3 2 5 1 0 7 3 5 9 2 7 9 8 9 7 6 9 jk Algorithm inPlacePartition(Sequence S, int j, int k, Entry pivot) while j < k do while C.compare(S.elemAtRank(j), pivot) < 0 do j  j + 1 while j < k and C.compare(S.elemAtRank(k), pivot) ≥ 0 do k  k – 1 if (j < k) then S.swapElement(j, k) // j contains the rank of the first element in G 3 2 5 1 0 2 3 5 9 7 7 9 8 9 7 6 9 j k j k

21. In-Place Quick-Sort Algorithm inPlaceQuickSort(Sequence S, int lower, int upper) if lower < upper then i  select a new pivot rank between lower and upper pivot  S.elemAtRank(i) remove pivot from S h  inPlacePartition(S, lower, upper-1, pivot) inPlaceQuickSort(S, lower, h - 1) inPlaceQuickSort(S, h, upper) S.insertAtRank(h, pivot) else // If lower == upper, we only have 1 Entry to sort // If lower > upper, then no Entries to sort. // So these are our base cases; we do not have to do anything! endif

22. Summary of Sorting Algorithms AlgorithmTimeNotes selection-sort O(n2)O(n2) uses priority queue slow (good for small inputs) insertion-sort O(n2)O(n2) uses priority queue slow (good for small inputs) quick-sort expected O(n log n) in-place, randomized fastest (good for large inputs) heap-sort O(n log n) uses additional heap fast (good for large inputs) merge-sort O(n log n) sequential data access fast (good for huge inputs)

23. Can We Do Better? Comparing entries to another we cannot do better than O ( n log n ) time We will talk more about this in a week But what if we had a sort that did not do not actually compare the entries Any ideas (from non-BIF400 students) how this would work?

24. Bucket-Sort (§ 10.4.1) Bucket-sort maintains auxiliary array, B, of Sequences (e.g., buckets) Bucket-sort sorts a Sequence, S, based upon the entry’s key: Phase 1: Move each entry in S to the end of bucket B[k], where k is the entry’s key Phase 2: For i  0, …, B.size ()-1, move entries in bucket B[i] to end of sequence S

25. Bucket-Sort Example Suppose keys range from [0, 9] 7, d1, c3, a7, g3, b7, e 1, c3, a3, b7, d7, g7, e Phase 1 Phase 2 012345689 B 1, c7, d7, g3, b3, a7, e  S 7   S

26. Bucket-Sort Algorithm Algorithm bucketSort(Sequence S, Comparator c) B  new Sequence[c.getMaxKey()] // instantiate the Sequence at each index within B while  S.isEmpty() do // Phase 1 entry  S.removeFirst() B[c.compare(entry, null)].insertLast(entry) for i  0 to B.length - 1 // Phase 2 while  B[i].isEmpty() do entry  B[i].removeFirst() S.insertLast(entry) return S

27. Bucket-Sort Properties Keys are indices into an array Must be non-negative integers and not arbitrary objects Does not need a Comparator Stable Sort Preserves relative ordering of two entries with same key Bubble-sort and Merge-sort can also be stable

28. Bucket-Sort Extensions Extend Bucket-sort with Comparator Specifies maximum number of buckets compare ( key, null ) returns index to which a key is mapped For Integer keys in range a – b: Comparator maps key k to k – a For Boolean keys, Comparator returns: index of 0 if key is false index of 1 if key is true

29. Bucket-Sort Extensions Can actually use Bucket-sort with any keys who can must be from some bounded set, D, of values E.g., D could be U.S. states, molecular structures, people I want to kill, … Comparator must have rank for each value in D  E.g., Rank states in alphabetical order or order of admission  Ranks then serve as the indicies

30. d -Tuples d- tuple is combination of d keys (k 1, k 2, …, k d ) k i is the “ i- th dimension of the tuple” Example: Points on a plane are a 2-tuple X-axis is 1 st dimension, Y-axis is 2 nd dimension

31. Lexicographic Order Lexicographic order of d- tuples is defined recursively as: (x 1, x 2, …, x d )  (y 1, y 2, …, y d )  x 1  y 1  ( x 1   y 1  (x 2, …, x d )  (y 2, …, y d )) How would we order following tuples? (3, 4) (7, 8) (3, 2) (1, 4) (4, 8) (1, 4) (3, 2) (3, 4) (4, 8) (7, 8)

32. Lexicographic Sorting Arranges d -tuples by making d calls to a stable sorting algorithm E.g., One call to sort each tuple dimension Concept lexicographicSort(Sequence s, Comparator c, Sort stableSort) for i  c.size() downto 1 stableSort.sort(s, c, i) return s

33. Lexicographic Sorting Example Example: (7,4,6) (5,1,5) (2,4,6) (2, 1, 4) (3, 2, 4)

34. Lexicographic Sorting Example Example: (7,4,6) (5,1,5) (2,4,6) (2,1,4) (3,2,4) (2,1,4) (3,2,4) (5,1,5) (7,4,6) (2,4,6) (2,1,4) (5,1,5) (3,2,4) (7,4,6) (2,4,6) (2,1,4) (2,4,6) (3,2,4) (5,1,5) (7,4,6)

35. Radix-Sort (§ 10.4.2) Radix-sort is a lexicographical sort that uses Bucket-sort as its stable sorting algorithm Radix-sort is applicable to tuples where the keys in each dimension i can be made into indexes in the range [0, N -1] Note: Integers are keys that can be considered to have multiple dimensions Pass i as compare ’s second parameter

36. Radix-Sort for Integers Can represent integer as a b -tuple of digits or bits: 62 = 111110 2 04 = 000100 2 Decimal representation uses need 10 buckets Binary numbers need 2 buckets Radix-sort algorithm runs in O(bn) time For 32-bit integers b = 32 Radix-sort sort Sequences using integer keys in O(32n) time

37. Radix-Sort for Integers Algorithm binaryRadixSort(Sequence S, Comparator c) for i  0 to 31 bucketSort(S, 2, i, c) return S If key is an int, the value of the i th bit of an integer is: (( key >> i ) & 1)

38. Example Sorting a sequence of 4-bit integers 1001 0010 1101 0001 1110 0010 1110 1001 1101 0001 1001 1101 0001 0010 1110 1001 0001 0010 1101 1110 0001 0010 1001 1101 1110

39. Daily Quiz Always selecting the first entry as the pivot, show the execution tree for Quick-sort when sorting the following in the order given Sequence with keys: 1 2 3 4 5 Sequence with keys: 5 4 3 2 1 Sequence with keys: 5 1 4 2 3