1. Design and Analysis of Algorithms – Chapter 51 Divide and Conquer (I) Dr. Ying Lu ylu@cse.unl.edu RAIK 283: Data Structures & Algorithms

2. Design and Analysis of Algorithms – Chapter 52 b Giving credit where credit is due: Most of the lecture notes are based on the slides from the Textbook’s companion websiteMost of the lecture notes are based on the slides from the Textbook’s companion website – http://www.aw-bc.com/info/levitin Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope CollegeSome examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College I have modified many of their slides and added new slides.I have modified many of their slides and added new slides. RAIK 283: Data Structures & Algorithms

3. Design and Analysis of Algorithms – Chapter 53 Divide and Conquer The most well known algorithm design strategy: 1. Divide instance of problem into two or more smaller instances 2. Solve smaller instances recursively 3. Obtain solution to original (larger) instance by combining these solutions

4. Design and Analysis of Algorithms – Chapter 54 Divide-and-conquer Technique subproblem 2 of size n/2 subproblem 1 of size n/2 a solution to subproblem 1 a solution to the original problem a solution to subproblem 2 a problem of size n

5. Design and Analysis of Algorithms – Chapter 55 Divide and Conquer Examples b Sorting: mergesort and quicksort b Tree traversals b Matrix multiplication-Strassen’s algorithm b Closest pair problem b Convex hull-QuickHull algorithm

6. Review Mergesort Design and Analysis of Algorithms – Chapter 56

7. 7 Mergesort Algorithm: b Split array A[1..n] in two and make copies of each half in arrays B[1.. n/2 ] and C[1.. n/2 ] b Sort arrays B and C b Merge sorted arrays B and C into array A

8. Design and Analysis of Algorithms – Chapter 58 Using Divide and Conquer: Mergesort b Mergesort Strategy Sorted Merge Sorted Sort recursively by Mergesort first last  (first  last)  2 

9. Design and Analysis of Algorithms – Chapter 59 Mergesort Algorithm: b Split array A[1..n] in two and make copies of each half in arrays B[1.. n/2 ] and C[1.. n/2 ] b Sort arrays B and C b Merge sorted arrays B and C into array A

10. Design and Analysis of Algorithms – Chapter 510 Mergesort Algorithm: b Split array A[1..n] in two and make copies of each half in arrays B[1.. n/2 ] and C[1.. n/2 ] b Sort arrays B and C b Merge sorted arrays B and C into array A as follows: Repeat the following until no elements remain in one of the arrays:Repeat the following until no elements remain in one of the arrays: –compare the first elements in the remaining unprocessed portions of the arrays –copy the smaller of the two into A, while incrementing the index indicating the unprocessed portion of that array Once all elements in one of the arrays are processed, copy the remaining unprocessed elements from the other array into A.Once all elements in one of the arrays are processed, copy the remaining unprocessed elements from the other array into A.

11. Design and Analysis of Algorithms – Chapter 511 Algorithm: Mergesort Input: Array E and indices first and last, such that the elements E[i] are defined for first  i  last. Output: E[first], …, E[last] is a sorted rearrangement of the same elements void mergeSort(Element[] E, int first, int last) if (first < last) int mid =  (first+last)/2  ; mergeSort(E, first, mid); mergeSort(E, mid+1, last); merge(E, first, mid, last); return;

12. Design and Analysis of Algorithms – Chapter 512 In-Class Exercise b P175 Problem 5.1.6 Apply Mergesort to sort the list E, X, A, M, P, L, E in alphabetical order.

13. Design and Analysis of Algorithms – Chapter 513 Algorithm: Mergesort void mergeSort(Element[] E, int first, int last) if (first < last) int mid =  (first+last)/2  ; mergeSort(E, first, mid); mergeSort(E, mid+1, last); merge(E, first, mid, last); return; Time Efficiency

14. Design and Analysis of Algorithms – Chapter 514 Mergesort Complexity b Mergesort always partitions the array equally. b Thus, the recursive depth is always  (lg n) b The amount of work done at each level is  (n) b Intuitively, the complexity should be  (n lg n) b We have, T(n) =2T(n/2)   (n) for n>1, T(1)=0   (n lg n)T(n) =2T(n/2)   (n) for n>1, T(1)=0   (n lg n) b The amount of extra memory used is  (n)

15. Design and Analysis of Algorithms – Chapter 515 Efficiency of Mergesort b Number of comparisons is close to theoretical minimum for comparison-based sorting: lg n ! ≈ n lg n - 1.44 n lg n ! ≈ n lg n - 1.44 n b Space requirement: Θ( n ) (NOT in-place) b Can be implemented without recursion (bottom-up) b All cases have same efficiency: Θ( n log n)

16. Design and Analysis of Algorithms – Chapter 516 Animation http://math.hws.edu/TMCM/java/xSortLab/index.html

17. Design and Analysis of Algorithms – Chapter 517 The master theorem b Given: a divide and conquer algorithm b Then, the Master Theorem gives us a cookbook for the algorithm’s running time:

18. Design and Analysis of Algorithms – Chapter 518 A general divide-and-conquer recurrence T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(n d ) T(1) = c 1. a < b d T(n) ∈ Θ(n d ) 2. a = b d T(n) ∈ Θ(n d lg n ) 3. a > b d T(n) ∈ Θ( n log b a ) Please refer to Appendix B for the proof

19. Design and Analysis of Algorithms – Chapter 519 In-class exercise b Page 175: Problem 5 b 5. Find the order of growth for solutions of the following recurrences. a. T(n) = 4T(n/2) + n, T(1) = 1a. T(n) = 4T(n/2) + n, T(1) = 1 b. T(n) = 4T(n/2) + n 2 2, T(1) = 1b. T(n) = 4T(n/2) + n 2 2, T(1) = 1 c. T(n) = 4T(n/2) + n 3 3, T(1) = 1c. T(n) = 4T(n/2) + n 3 3, T(1) = 1

20. In-Class Exercise b 5.1.9 Let A[0, n-1] be an array of n distinct real numbers. A pair (A[i], A[j]) is said to be an inversion if these numbers are out of order, i.e., i A[j]. Design an O(nlogn) algorithm for counting the number of inversions. Design and Analysis of Algorithms – Chapter 520

21. Review Quicksort Design and Analysis of Algorithms – Chapter 521

22. Design and Analysis of Algorithms – Chapter 522 Quicksort by Hoare (1962) b Select a pivot (partitioning element) b Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot b Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final position b Sort the two sublists recursively p A[i]≤p A[i]  p

23. Design and Analysis of Algorithms – Chapter 523 Quicksort by Hoare (1962) b Select a pivot (partitioning element) b Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot b Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final position b Sort the two sublists recursively b Apply quicksort to sort the list 7 2 9 10 5 4 p A[i]≤p A[i]  p

24. Design and Analysis of Algorithms – Chapter 524 Quicksort Example b Recursive implementation with the left most array entry selected as the pivot element.

25. Design and Analysis of Algorithms – Chapter 525 Quicksort Quicksort Animated Example: http://math.hws.edu/TMCM/java/xSortLab/index.html http://math.hws.edu/TMCM/java/xSortLab/index.html

26. Design and Analysis of Algorithms – Chapter 526 Quicksort Algorithm b Input: bArray E and indices first, and last, s.t. elements E[i] are defined for first  i  last b Ouput: bE[first], …, E[last] is a sorted rearrangement of the array b Void quickSort(Element[] E, int first, int last) if (first < last) Element pivotElement = E[first]; int splitPoint = partition(E, pivotElement, first, last); quickSort (E, first, splitPoint –1 ); quickSort (E, splitPoint +1, last ); return;

27. Design and Analysis of Algorithms – Chapter 527 Quicksort Analysis b Partition can be done in O(n) time, where n is the size of the array b Let T(n) be the number of comparisons required by Quicksort b If the pivot ends up at position k, then we have T(n)  T(n  k)  T(k  1)  nT(n)  T(n  k)  T(k  1)  n b To determine best-, worst-, and average-case complexity we need to determine the values of k that correspond to these cases.

28. Design and Analysis of Algorithms – Chapter 528 Quicksort Analysis b Partition can be done in O(n) time, where n is the size of the array b Let T(n) be the number of comparisons required by Quicksort b If the pivot ends up at position k, then we have T(n)  T(n  k)  T(k  1)  nT(n)  T(n  k)  T(k  1)  n b To determine best-, worst-, and average-case complexity we need to determine the values of k that correspond to these cases.

29. Design and Analysis of Algorithms – Chapter 529 Best-Case Complexity b The best case is clearly when the pivot always partitions the array equally. b Intuitively, this would lead to a recursive depth of at most lg n calls b We can actually prove this. In this case T(n)  T(n/2)  T(n/2)  n   (n lg n)T(n)  T(n/2)  T(n/2)  n   (n lg n)

30. Design and Analysis of Algorithms – Chapter 530 Worst-Case and Average-Case Complexity b The worst-case is when the pivot always ends up in the first or last element. That is, partitions the array as unequally as possible. b In this case T(n)  ?T(n)  ?

31. Design and Analysis of Algorithms – Chapter 531 Worst-Case and Average-Case Complexity b The worst-case is when the pivot always ends up in the first or last element. That is, partitions the array as unequally as possible. b In this case T(n)  T(n  1)  T(1  1)  n  T(n  1)  nT(n)  T(n  1)  T(1  1)  n  T(n  1)  n  n  (n  1)  … + 1  n(n  1)/2   (n 2 ) b Average case is rather complex, but is where the algorithm earns its name. The bottom line is:

32. Design and Analysis of Algorithms – Chapter 532 Summary of Quicksort b Best case: split in the middle — Θ( n log n) b Worst case: sorted array! — Θ( n 2 ) b Average case: random arrays — Θ( n log n)

33. Design and Analysis of Algorithms – Chapter 533 Summary of Quicksort b Best case: split in the middle — Θ( n log n) b Worst case: sorted array! — Θ( n 2 ) b Average case: random arrays — Θ( n log n) b Memory requirement?

34. Design and Analysis of Algorithms – Chapter 534 Summary of Quicksort b Best case: split in the middle — Θ( n log n) b Worst case: sorted array! — Θ( n 2 ) b Average case: random arrays — Θ( n log n) b Memory requirement? b In-place sorting algorithm b Considered as the method of choice for internal sorting for large files (n ≥ 10000)

35. Design and Analysis of Algorithms – Chapter 535 Summary of Quicksort b Best case: split in the middle — Θ( n log n) b Worst case: sorted array! — Θ( n 2 ) b Average case: random arrays — Θ( n log n) b Considered as the method of choice for internal sorting for large files (n ≥ 10000) b Improvements: better pivot selection: median of three partitioning avoids worst case in sorted filesbetter pivot selection: median of three partitioning avoids worst case in sorted files switch to insertion sort on small subfilesswitch to insertion sort on small subfiles elimination of recursionelimination of recursion these combine to 20-25% improvement

36. In-class exercises Design and Analysis of Algorithms – Chapter 536