1. Lecture 5: Linear Time Sorting Shang-Hua Teng

2. Sorting Input: Array A[1...n], of elements in arbitrary order; array size n Output: Array A[1...n] of the same elements, but in the non-decreasing order

3. How fast can we sort? Can we do better than O(n log n)? Well… it depends? It depends on the model of computation!

4. Comparison Sorting Only comparisons are used to determine the order of elements Example –Insertion sort –Merge Sort –Quick Sort –Heapsort

5. Decision Trees Example: 3 element sorting (a 1, a 2, a 3 ) a 1 ? a 2 “Less than” leads to left branch “Great than” leads to right branch

6. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2

7. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1, a 2, a 3

8. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1 ? a 3 a 1, a 2, a 3

9. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1 ? a 3 a 1, a 2, a 3 a 3, a 1, a 2 a 1, a 3, a 2

10. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1 ? a 3 a 1, a 2, a 3 a 3, a 1, a 2 a 1, a 3, a 2

11. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1 ? a 3 a 1, a 2, a 3 a 3, a 1, a 2 a 1, a 3, a 2 a 2, a 1, a 3

12. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1 ? a 3 a 2 ? a 3 a 1 ? a 3 a 1, a 2, a 3 a 3, a 1, a 2 a 1, a 3, a 2 a 2, a 1, a 3 a 2, a 3, a 1 a 3, a 2, a 1

13. Decision Trees Example: 3 element sort (a 1, a 2, a 3 ) a 2 ? a 3 a 1 ? a 2 a 1 ? a 3 a 2 ? a 3 a 1 ? a 3 a 1, a 2, a 3 a 3, a 1, a 2 a 1, a 3, a 2 a 2, a 1, a 3 a 2, a 3, a 1 a 3, a 2, a 1

14. Decision Trees Each leaf contains a permutation indicating that permutation of order has been established A decision tree can model a comparison sort It is the tree of all possible instruction traces Running time = length of path Worst case running time = length of the longest path (height of the tree)

15. Lower Bound for Comparison Sorting Theorem: Any decision tree that sorts n elements has height  (n lg n) Proof: There are n! leaves A binary tree of height h has no more than 2 h nodes

16. Optimal Comparison Sorting Algorithms Merge sort Quick sort Heapsort

17. Sorting in Linear Time Counting sort –Suppose we only have k keys {1,2,…,k} –  ( n+k ) time Radix sort –Can use counting sort –Place by place (digit by digit ) sorting –IBM Human machine interface –If elements have d places (digits), then we need d calls to counting sort

18. Counting Sort Counts how many occurrences of each key Allocate proper amount of spaces for each key, from small to large For each element a[j] in the input, count the number of elements that are less than the element A[k] is “Less than” A[j] if –A[k] < A[j] or –A[k] = A[j] and k < j

19. Example of Counting Sort A = [4 1 3 4 3], we have four keys C =[1 0 2 2] Prefix sum of C –Prefix_sum( C ) = 0 [1 1 3 5] –First key goes from 1 to 1 –Second key goes from 2 to 1 (empty) –Third key goes from 2 to 3 –Fourth key goes from 4 5 Sorted array is = [1 3 3 4 4]

20. Counting Sort Counting_sort (A, 1, n) –for i = 0 to k do –for j = 1 to length(A) do –for i = 1 to k do –for j = length(A) downto 1 do Zero out the counting array C[j] now contains the number of elements equal to i C[j] now contains the number of elements less than or equal to j

21. Stable sorting Stable: after sorting elements with the same values appear in the output array in the same order as they do in the input array An important property of counting sort –It is stable

22. Complexity of Counting Sort  (n+k)

23. Radix Sort 329 457 657 839 436 720 355

24. Radix Sort 329 457 657 839 436 720 355 720 355 436 457 657 329 839

25. Radix Sort 329 457 657 839 436 720 355 720355436457657329839720355436457657329839 720329436839355457657720329436839355457657

26. Radix Sort 329 457 657 839 436 720 355 720 355 436 457 657 329 839 720 329 436 839 355 457 657 329 355 436 457 657 720 839

27. Correctness Induction on digit position –Assume numbers are sorted by low-order t-1 digits –Sort on digit t Two numbers that differ in digit t are correctly sorted 2 number = in digit t are put in same order as input, implying correct order Spreadsheets use stable sort, so you can try by hand if you have spreadsheets

28. Time Complexity of Radix sort Sort n words with b bits each –b passes of counting sort on 1-bits digits –Or b/2 passes … 2-bits digits –Or b/r passes … r-bits digits r = log n Keys from {1,…,n d } needs  (dn) time