1. Tirgul 4 Subjects of this Tirgul: Counting Sort Radix Sort Bucket Sort

2. Linear-time Sorting The lower-bound running time for comparison sorting algorithms is The catch cannot use anything but pair wise comparisons No prior information on the elements compared With more information we can do better

3. Counting Sort The input – n integers in the range [0,k-1] (k is an integer) When k = O(n), the total running time of the sort is also O(n) We use three arrays for the sort: in[0..n-1]-The input array out[0..n-1]-The output array C[0..k-1]-A temporary array (the counting array)

4. Counting Sort The idea For each element x, how many elements are less than x? If there are 6 elements less than x, it belongs in the 7 th position (in the output array) If there are several elements with the same value we locate them one after the other.

5. Counting Sort public static void CountingSort(int[] in, int[] out, int k) { int[] c = new int[k]; // now c[i] = 0 for all i for (int i = 0; i<in.length ; i++) c[in[i]] ++; // now c[i-1] holds the number of elements in the input equal to i for (int i = 1 ; i<c.length ; i++ ) c[i] += c[i-1]; // now c[i-1] holds the number of elements in the input which are <= i for (int i=in.length-1 ; i>=0 ; i--) { out[ c[in[i]] -1 ] = in[i]; c[in[i]] --; //We make sure that equal elements will be located one after the other. }

6. Counting Sort - example First loop in: 3 6 4 1 3 4 1 4 C: 0 2 0 2 3 0 1 Second loop C: 0 2 2 4 7 7 8 Third loop: first cycle: out: 0 0 0 0 0 0 4 0 C: 0 2 2 4 6 7 8 final cycle: out: 1 1 3 3 4 4 4 6 C: 0 0 2 2 4 7 7

7. Counting Sort Running time The first for loop runs in O(n) The second for loop runs in O(k) The third for loop runs in O(n) The total run time O(n+ k) If k = O(n) the total run time will be O(n). We gain run time. Counting sort ’ s run time is shorter than any comparison sort. Advantage: stable Disadvantage: requires additional memory (not in place)

8. Radix Sort Some history … Once upon a time in order to compile their programs the poor programmers had to punch cards. They had to punch numbers with 80 digits, each digit in base 12. The compilation process was operated by a miserable guy called the operator. This operator had to feed ordered cards to the computer. Unfortunately he received the cards in messy jumbles from the programmers. One day, in his desperation, the poor operator invented Radix Sort.

9. Radix Sort Input: d-digit numbers, each digit in the range 0..k-1. First intuition: sort from most significant to least significant digit. Recursively sort every sub pile. The problem: many sub piles to keep track of when the recursion done. Radix sort idea: sort from least significant to most significant digit. each time by one digit only. Use stable sort each time.

10. Radix Sort RADIX-SORT(A,d) for i 1 to d do use a stable sort to sort array A on digit i When every digit is in the range [0,k-1] and k is not too big we use Counting Sort. Running time: For every digit O(n+ k) For d digits O(dn+ dk) Total run time, when d is a constant and k = O(n), is O(n).

11. Radix Sort Why does it work? The trick is by using the stable sort. Example: From least significant digit 839657839355 720457329720 657355657436 457839457839 436 657 355329355457 329720 329

12. Radix Sort Prove that the Radix Sort works properly. Proof by induction. Definition for the proof: For any number x in the input, define x i as the number excepted after considering only the i list significant digits of the number x. For example x =531 x 2 =31

13. Radix Sort We claim that after i passages of the algorithm the group of numbers xi is sorted. Induction base: i =1 If xi<yi x<y (we have only one digit) Assume that the Radix Sort works properly until iteration i-1 and prove the claim for the i ’ th iteration.

14. Radix Sort Assume that xi<yi. There are two options. The i ’ th digit of x < than the i ’ th digit of y. The i ’ th digit of x and y are equal. (xi<yi do to the previews i-1 digits) In the first case i ’ th iteration sort according to the i ’ th digit Since we assume xi-1<yi-1 after the i-1 ’ th iteration and the i ’ th iteration sort according to the i ’ th digit using a stable sort xi<yi

15. Questions How to sort n integers in the range 1 to n 2 in O(n) time? Suppose you have an array of n data records to sort and that the key of each record has the value 0/1. Give a linear time algorithm for sorting these records with no extra space. Can the algorithm from the previous question be applied to radix sort n records with b-bits keys in O(b*n) time?

16. Bucket Sort The input – n real numbers in the interval [0,1) (i.e. ), distributed uniformly The running time is O(n) on the average The idea Divide the interval into n ‘ buckets ’. Put each element into its matching bucket. As the numbers are uniformly distributed, not too many elements will be placed in each bucket – so use insertion sort to sort each bucket, and then concatenate the contents into a single list

17. Bucket Sort Running time: The first loop and last step (concatenation) are O(n). The insertion sorts: Denote by n i the number of elements in bucket i. The expected time to sort bucket i is therefore The total An element fits into each bucket with probability 1/n so P(n i =k) is binomial:

18. Bucket Sort The total average running time of the sort is O(n)