Design and Analysis of Algorithms Non-comparison sort (sorting in linear time) Haidong Xue Summer 2012, at GSU

Comparison based sorting Algorithms that determine sorted order based only on comparisons between the input elements AlgorithmWorst TimeExpected TimeExtra MemoryStable Insertion sortO(1) (in place)Can be Merge sortO(n)Can be Quick sortO(1) (in place)Can be Heap sortO(1) (in place)No What is the lower bound?

Lower bounds for comparison based sorting <? Y N Y …… ………………………….. Done N Y For n element array, how many possible inputs are there? Factorial of n n! What is the shortest tree can have n! leaves? As a result ……

Sorting in linear time Can we sort an array in linear time? Yes, but not for free E.g. sort cards with 13 slots What if there are more than one elements in the same slot?

Counting Sort Input: array A[1, …, n]; k (elements in A have values from 1 to k) Output: sorted array A Algorithm: 1.Create a counter array C[1, …, k] 2.Create an auxiliary array B[1, …, n] 3.Scan A once, record element frequency in C 4.Calculate prefix sum in C 5.Scan A in the reverse order, copy each element to B at the correct position according to C. 6.Copy B to A

Counting Sort A: C: B: Position indicator:

Analysis of Counting Sort Input: array A[1, …, n]; k (elements in A have values from 1 to k) Output: sorted array A Algorithm: 1.Create a counter array C[1, …, k] 2.Create an auxiliary array B[1, …, n] 3.Scan A once, record element frequency in C 4.Calculate prefix sum in C 5.Scan A in the reverse order, copy each element to B at the correct position according to C. 6.Copy B to A Time O(n) O(k) O(n) O(n+k)=O(n) (if k=O(n)) Space O(k) O(n) O(n+k)=O(n) (if k=O(n))

Radix-Sort Input: array A[1, …, n]; d (number of digit a element has) Output: sorted array A Algorithm: for each digit{ use a stable sort to sort A on a digit } T(n)=O(d(n+k))

Summary AlgorithmWorst TimeExpected TimeExtra MemoryStable Insertion sortO(1) (in place)Yes Merge sortO(n)Yes Quick sortO(1) (in place)Yes Heap sortO(1) (in place)No Counting sortYes Design strategies: Divide and conquer Employ certain special data structure Tradeoff between time and space

Knowledge tree Algorithms Analysis Design Algorithms for classic problems Classic data structure Asymptotic notations Probabilisti c analysis Sorting Shortest path Matrix multiplication … Divide & Conquer Greedy Dynamic Programming O(), o(),  (),  (),  () Heap, Hashing, Binary Tree, RBT, …. Quicksort, Heapsort, Mergesort, … ……… … … … ………… … … … … … ……… …