Presentation on theme: "CSE 3101: Introduction to the Design and Analysis of Algorithms"— Presentation transcript:
1CSE 3101: Introduction to the Design and Analysis of Algorithms Suprakash Dattadatta[at]cse.yorku.ca4/13/2017CSE 31011
2Characteristics Quick Sort sorts ”almost” in place, i.e., does not require an additional array, like insertion sortDivide-and-conquer, like merge sortvery practical, average sort performance O(n log n) (with small constant factors), but worst case O(n2) [CAVEAT: this is true for the CLRS version]
3Quick Sort – the main idea To understand quick-sort, let’s look at a high-level description of the algorithmA divide-and-conquer algorithmDivide: partition array into 2 subarrays such that elements in the lower part <= elements in the higher partConquer: recursively sort the 2 subarraysCombine: trivial since sorting is done in place
4Linear time partitioning procedure jPartition(A,p,r)01 x¬A[r]02 i¬p-103 j¬r+104 while TRUE05 repeat j¬j-106 until A[j] £x07 repeat i¬i+108 until A[i] ³x09 if i<j10 then exchange A[i]«A[j]11 else return j1712619238510£ X=10 £ij1012619238517ij1056192381217ji10568231912171056823191217
5Initial call Quicksort(A, 1, length[A]) Quick Sort AlgorithmInitial call Quicksort(A, 1, length[A])Quicksort(A,p,r)01 if p<r02 then q¬Partition(A,p,r)03 Quicksort(A,p,q)04 Quicksort(A,q+1,r)
6Assume that all input elements are distinct Analysis of QuicksortAssume that all input elements are distinctThe running time depends on the distribution of splits
7Best CaseIf we are lucky, Partition splits the array evenly
8Using the median as a pivot The recurrence in the previous slide works out, BUT……Q: Can we find the median in linear-time?A: Yes! Chapter 9 of the textNote : Most implementations do not use the median as pivot.
9One side of the parition has only one element Worst CaseWhat is the worst case?One side of the parition has only one element
11When does the worst case appear? input is sortedinput reverse sortedSame recurrence for the worst case of insertion sortHowever, sorted input yields the best case for insertion sort!
12Suppose the split is 1/10 : 9/10 Analysis of QuicksortSuppose the split is 1/10 : 9/10
13An Average Case Scenario Suppose, we alternate lucky and unlucky cases to get an average behaviornnlarger constant in the O notationn-11(n-1)/2(n-1)/2(n-1)/2+1(n-1)/2
14An Average Case Scenario (2) How can we make sure that we are usually lucky?Partition around the ”middle” (n/2th) element?Partition around a random element (works well in practice)Randomized algorithmrunning time is independent of the input orderingno specific input triggers worst-case behaviorthe worst-case is only determined by the output of the random-number generator
15Assume all elements are distinct Partition around a random element Randomized QuicksortAssume all elements are distinctPartition around a random elementRandomization is often used to design algorithms with good average-case complexity (the worst-case complexity may not be as good)
16The optimality question Q: Can we do better that worst case Q(n log n) time for sorting?A: In general no, but in some special cases yes!Q: Why not?A: The well-known (n log n) lower bound.
17“the best any algorithm can do” for a problem On Lower Bounds“the best any algorithm can do” for a problemThe proof must be algorithm independentIn general, lower bound proofs are difficultMust make some assumptions – the sorting lower bound assumes that sorting is comparison based.This will be covered later today, or by Prof. Ruppert next weekIf we relax the “comparison-based” assumption, we can sort in linear time!
18Q: How we beat the (n log n) lower bound for sorting? Next: Linear sortingQ: How we beat the (n log n) lower bound for sorting?A: By making extra assumptions about the input
19Non-Comparison Sort – Bucket Sort Assumption: uniform distributionInput numbers are uniformly distributed in [0,1).Suppose input size is n.Idea:Divide [0,1) into n equal-sized subintervals (buckets).Distribute n numbers into bucketsExpect that each bucket contains few numbers.Sort numbers in each bucket (insertion sort as default).Then go through buckets in order, listing elementsCan be shown to run in linear-time on average
21Generalizing Bucket Sort Q: What if the input numbers are NOT uniformly distributed in [0,1)?A: Can be generalized in different ways, e.g. if the distribution is known we can design (unequal sized) bins that will have roughly equal number of numbers on average.
22Non-Comparison Sort – Counting Sort Assumption: n input numbers are integers in the range [0,k], k=O(n).Idea:Determine the number of elements less than x, for each input x.Place x directly in its position.
23Counting Sort - pseudocode Counting-Sort(A,B,k)for i0 to kdo C[i] 0for j 1 to length[A]do C[A[j]] C[A[j]]+1// C[i] contains number of elements equal to i.for i 1 to kdo C[i]=C[i]+C[i-1]// C[i] contains number of elements i.for j length[A] downto 1do B[C[A[j]]] A[j]C[A[j]] C[A[j]]-1
25Counting Sort - analysis for i0 to k (k)do C[i] (1)for j 1 to length[A] (n)do C[A[j]] C[A[j]] (1) ((1) (n)= (n))// C[i] contains number of elements equal to i. (0)for i 1 to k (k)do C[i]=C[i]+C[i-1] (1) ((1) (n)= (n))// C[i] contains number of elements i (0)for j length[A] downto (n)do B[C[A[j]]] A[j] (1) ((1) (n)= (n))C[A[j]] C[A[j]] (1) ((1) (n)= (n))Total cost is (k+n), suppose k=O(n), then total cost is (n).So, it beats the (n log n) lower bound!
26Preserves order of elements with the same key. Stable sortPreserves order of elements with the same key.Counting sort is stable.Crucial question: can counting sort be used to sort large integers efficiently?
27do use a stable sort to sort A on digit i Analysis: Radix sortRadix-Sort(A,d)for i1 to ddo use a stable sort to sort A on digit iAnalysis:Given n d-digit numbers where each digit takes on up to k values, Radix-Sort sorts these numbers correctly in (d(n+k)) time.