1. Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu

2. CS 477/6772 Divide-and-Conquer Divide the problem into a number of subproblems –Similar sub-problems of smaller size Conquer the sub-problems –Solve the sub-problems recursively –Sub-problem size small enough  solve the problems in straightforward manner Combine the solutions to the sub-problems –Obtain the solution for the original problem

3. CS 477/6773 Analyzing Divide-and Conquer Algorithms The recurrence is based on the three steps of the paradigm: –T(n) – running time on a problem of size n –Divide the problem into a subproblems, each of size n/b: takes D(n) –Conquer (solve) the subproblems aT(n/b) –Combine the solutions C(n)  (1) if n ≤ c T(n) = aT(n/b) + D(n) + C(n) otherwise

4. CS 477/6774 Merge Sort Approach To sort an array A[p.. r]: Divide –Divide the n-element sequence to be sorted into two subsequences of n/2 elements each Conquer –Sort the subsequences recursively using merge sort –When the size of the sequences is 1 there is nothing more to do Combine –Merge the two sorted subsequences

5. CS 477/6775 Merge Sort - Discussion Running time insensitive of the input Advantages: –Guaranteed to run in  (nlgn) Disadvantage –Requires extra space  N

6. CS 477/6776 Quicksort Sort an array A[p…r] Divide –Partition the array A into 2 subarrays A[p..q] and A[q+1..r], such that each element of A[p..q] is smaller than or equal to each element in A[q+1..r] –The index (pivot) q is computed Conquer –Recursively sort A[p..q] and A[q+1..r] using Quicksort Combine –Trivial: the arrays are sorted in place  no work needed to combine them: the entire array is now sorted A[p…q]A[q+1…r] ≤

7. CS 477/6777 QUICKSORT Alg.: QUICKSORT (A, p, r) if p < r then q  PARTITION (A, p, r) QUICKSORT (A, p, q) QUICKSORT (A, q+1, r)

8. CS 477/6778 Partitioning the Array Idea –Select a pivot element x around which to partition –Grows two regions A[p…i]  x x  A[j…r] A[p…i]  xx  A[j…r] ij

9. CS 477/6779 Example 73146235 ij 75146233 ij 75146233 ij 75641233 ij 73146235 ij A[p…r] 75641233 ij A[p…q]A[q+1…r]

10. CS 477/67710 Partitioning the Array Alg. PARTITION (A, p, r) 1. x  A[p] 2. i  p – 1 3. j  r + 1 4. while TRUE 5. do repeat j  j – 1 6. until A[j] ≤ x 7. repeat i  i + 1 8. until A[i] ≥ x 9. if i < j 10. then exchange A[i]  A[j] 11. else return j Running time:  (n) n = r – p + 1 73146235 i j A: arar apap ij=q A: A[p…q]A[q+1…r] ≤ p r

11. CS 477/67711 Performance of Quicksort Worst-case partitioning –One region has one element and one has n – 1 elements –Maximally unbalanced Recurrence T(n) = T(n – 1) +  (n), T(1) =  (1) T(n) = T(n – 1) +  (n) = n n - 1 n - 2 n - 3 2 1 1 1 1 1 1 n n n n - 1 n - 2 3 2  (n 2 )

12. CS 477/67712 Performance of Quicksort Best-case partitioning –Partitioning produces two regions of size n/2 Recurrence T(n) = 2T(n/2) +  (n) T(n) =  (nlgn) (Master theorem)

13. CS 477/67713 Performance of Quicksort Balanced partitioning –Average case closer to best case than worst case –Partitioning always produces a constant split E.g.: 9-to-1 proportional split T(n) = T(9n/10) + T(n/10) + n

14. CS 477/67714 Performance of Quicksort Average case –All permutations of the input numbers are equally likely –On a random input array, we will have a mix of well balanced and unbalanced splits –Good and bad splits are randomly distributed across throughout the tree Alternate of a good and a bad split Nearly well balanced split n n - 1 1 (n – 1)/2 n (n – 1)/2 + 1 Running time of Quicksort when levels alternate between good and bad splits is O(nlgn) combined cost: 2n-1 =  (n) combined cost: n =  (n)

15. CS 477/67715 Randomizing Quicksort Randomly permute the elements of the input array before sorting Modify the PARTITION procedure –At each step of the algorithm we exchange element A[p] with an element chosen at random from A[p…r] –The pivot element x = A[p] is equally likely to be any one of the r – p + 1 elements of the subarray

16. CS 477/67716 Randomized Algorithms The behavior is determined in part by values produced by a random-number generator –RANDOM (a, b) returns an integer r, where a ≤ r ≤ b and each of the b-a+1 possible values of r is equally likely Algorithm generates its own randomness No input can elicit worst case behavior –Worst case occurs only if we get “unlucky” numbers from the random number generator

17. CS 477/67717 Randomized PARTITION Alg.: RANDOMIZED-PARTITION (A, p, r) i ← RANDOM (p, r) exchange A[p] ↔ A[i] return PARTITION (A, p, r)

18. CS 477/67718 Randomized Quicksort Alg. : RANDOMIZED-QUICKSORT (A, p, r) if p < r then q ← RANDOMIZED-PARTITION (A, p, r) RANDOMIZED-QUICKSORT (A, p, q) RANDOMIZED-QUICKSORT (A, q + 1, r)

19. CS 477/67719 Worst-Case Analysis of Quicksort T(n) = worst-case running time T(n) = max (T(q) + T(n-q)) +  (n) 1 ≤ q ≤ n-1 Use substitution method to show that the running time of Quicksort is O(n 2 ) Guess T(n) = O(n 2 ) –Induction goal: T(n) ≤ cn 2 –Induction hypothesis: T(k) ≤ ck 2 for any k ≤ n

20. CS 477/67720 Worst-Case Analysis of Quicksort Proof of induction goal: T(n) ≤ max (cq 2 + c(n-q) 2 ) +  (n) = 1 ≤ q ≤ n-1 = c  max (q 2 + (n-q) 2 ) +  (n) 1 ≤ q ≤ n-1 The expression q 2 + (n-q) 2 achieves a maximum over the range 1 ≤ q ≤ n-1 at one of the endpoints max (q 2 + (n - q) 2 ) ≤ 1 2 + (n - 1) 2 = n 2 – 2(n – 1) 1 ≤ q ≤ n-1 T(n) ≤ cn 2 – 2c(n – 1) +  (n) ≤ cn 2

21. CS 477/67721 Another Way to PARTITION Given an array A, partition the array into the following subarrays: –A pivot element x = A[q] –Subarray A[p..q-1] such that each element of A[p..q-1] is smaller than or equal to x (the pivot) –Subarray A[q+1..r], such that each element of A[p..q+1] is strictly greater than x (the pivot) Note: the pivot element is not included in any of the two subarrays A[p…i] ≤ xA[i+1…j-1] > x pii+1 rj-1 unknown pivot j

22. CS 477/67722 Example

23. CS 477/67723 Another Way to PARTITION Alg.: PARTITION(A, p, r) x ← A[r] i ← p - 1 for j ← p to r - 1 do if A[ j ] ≤ x then i ← i + 1 exchange A[i] ↔ A[j] exchange A[i + 1] ↔ A[r] return i + 1 Chooses the last element of the array as a pivot Grows a subarray [p..i] of elements ≤ x Grows a subarray [i+1..j-1] of elements >x Running Time:  (n), where n=r-p+1 A[p…i] ≤ xA[i+1…j-1] > x pii+1 rj-1 unknown pivot j

24. CS 477/67724 Loop Invariant 1.All entries in A[p.. i] are smaller than the pivot 2.All entries in A[i + 1.. j - 1] are strictly larger than the pivot 3.A[r] = pivot 4.A[ j.. r -1] elements not yet examined A[p…i] ≤ xA[i+1…j-1] > x pii+1 rj-1 x unknown pivot

25. CS 477/67725 Readings Chapter 7