Presentation on theme: "15/1/2015CSE 31011 Suprakash Datta datta[at]cse.yorku.ca CSE 3101: Introduction to the Design and Analysis of Algorithms."— Presentation transcript:
15/1/2015CSE Suprakash Datta datta[at]cse.yorku.ca CSE 3101: Introduction to the Design and Analysis of Algorithms
2 Quick Sort Characteristics –sorts ” almost ” in place, i.e., does not require an additional array, like insertion sort –Divide-and-conquer, like merge sort –very practical, average sort performance O(n log n) (with small constant factors), but worst case O(n 2 ) [CAVEAT: this is true for the CLRS version]
3 Quick Sort – the main idea To understand quick-sort, let ’ s look at a high- level description of the algorithm A divide-and-conquer algorithm –Divide: partition array into 2 subarrays such that elements in the lower part <= elements in the higher part –Conquer: recursively sort the 2 subarrays –Combine: trivial since sorting is done in place
4 Partitioning Linear time partitioning procedure Partition(A,p,r) 01 x A[r] 02 i p-1 03 j r+1 04 while TRUE 05 repeat j j-1 06 until A[j] x 07 repeat i i+1 08 until A[i] x 09 if i
5 Quick Sort Algorithm Initial call Quicksort(A, 1, length[A]) Quicksort(A,p,r) 01 if p
6 Analysis of Quicksort Assume that all input elements are distinct The running time depends on the distribution of splits
7 Best Case If we are lucky, Partition splits the array evenly
8 Using the median as a pivot The recurrence in the previous slide works out, BUT …… Q: Can we find the median in linear-time? A: Yes! Chapter 9 of the text Note : Most implementations do not use the median as pivot.
9 Worst Case What is the worst case? One side of the parition has only one element
10 Worst Case (2)
11 Worst Case (3) When does the worst case appear? –input is sorted –input reverse sorted Same recurrence for the worst case of insertion sort However, sorted input yields the best case for insertion sort!
12 Analysis of Quicksort Suppose the split is 1/10 : 9/10
13 An Average Case Scenario Suppose, we alternate lucky and unlucky cases to get an average behavior n 1 n-1 (n-1)/2 (n-1)/2+1 (n-1)/2 n
14 An Average Case Scenario (2) How can we make sure that we are usually lucky? –Partition around the ” middle ” (n/2th) element? –Partition around a random element (works well in practice) Randomized algorithm –running time is independent of the input ordering –no specific input triggers worst-case behavior –the worst-case is only determined by the output of the random-number generator
15 Randomized Quicksort Assume all elements are distinct Partition around a random element Randomization is often used to design algorithms with good average-case complexity (the worst-case complexity may not be as good)
16 The optimality question Q: Can we do better that worst case (n log n) time for sorting? A: In general no, but in some special cases yes! Q: Why not? A: The well-known (n log n) lower bound.
17 On Lower Bounds “ the best any algorithm can do” for a problem The proof must be algorithm independent In general, lower bound proofs are difficult Must make some assumptions – the sorting lower bound assumes that sorting is comparison based. This will be covered later today, or by Prof. Ruppert next week If we relax the “comparison-based” assumption, we can sort in linear time!
18 Next: Linear sorting Q: How we beat the (n log n) lower bound for sorting? A: By making extra assumptions about the input
19 Non-Comparison Sort – Bucket Sort Assumption: uniform distribution –Input numbers are uniformly distributed in [0,1). –Suppose input size is n. Idea: –Divide [0,1) into n equal-sized subintervals (buckets). –Distribute n numbers into buckets –Expect that each bucket contains few numbers. –Sort numbers in each bucket (insertion sort as default). –Then go through buckets in order, listing elements Can be shown to run in linear-time on average
20 Example of BUCKET-SORT
21 Generalizing Bucket Sort Q: What if the input numbers are NOT uniformly distributed in [0,1)? A: Can be generalized in different ways, e.g. if the distribution is known we can design (unequal sized) bins that will have roughly equal number of numbers on average.
22 Non-Comparison Sort – Counting Sort Assumption: n input numbers are integers in the range [0,k], k=O(n). Idea: –Determine the number of elements less than x, for each input x. –Place x directly in its position.
23 Counting Sort - pseudocode Counting-Sort(A,B,k) for i 0 to k do C[i] 0 for j 1 to length[A] do C[A[j]] C[A[j]]+1 // C[i] contains number of elements equal to i. for i 1 to k do C[i]=C[i]+C[i-1] // C[i] contains number of elements i. for j length[A] downto 1 do B[C[A[j]]] A[j] C[A[j]] C[A[j]]-1
24 Counting Sort - example
25 Counting Sort - analysis 1.for i 0 to k (k) 2. do C[i] 0 (1) 3.for j 1 to length[A] (n) 4. do C[A[j]] C[A[j]]+1 (1) ( (1) (n)= (n)) 5.// C[i] contains number of elements equal to i. (0) 6.for i 1 to k (k) 7. do C[i]=C[i]+C[i-1] (1) ( (1) (n)= (n)) 8.// C[i] contains number of elements i. (0) 9.for j length[A] downto 1 (n) 10. do B[C[A[j]]] A[j] (1) ( (1) (n)= (n)) 11. C[A[j]] C[A[j]]-1 (1) ( (1) (n)= (n)) Total cost is (k+n), suppose k=O(n), then total cost is (n). So, it beats the (n log n) lower bound!
26 Stable sort Preserves order of elements with the same key. Counting sort is stable. Crucial question: can counting sort be used to sort large integers efficiently?
27 Radix sort Radix-Sort(A,d) for i 1 to d do use a stable sort to sort A on digit i Analysis: Given n d-digit numbers where each digit takes on up to k values, Radix-Sort sorts these numbers correctly in (d(n+k)) time.