1. Divide-and-Conquer1 7 2  9 4  2 4 7 9 7  2  2 79  4  4 9 7  72  29  94  4

2. Divide-and-Conquer2 Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two or more disjoint subsets S 1, S 2, … Recur: solve the subproblems recursively Conquer: combine the solutions for S 1, S 2, …, into a solution for S The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations

3. Divide-and-Conquer3 Merging Two Sorted Sequences The conquer step of merge-sort consists of merging two sorted sequences A and B into a sorted sequence S containing the union of the elements of A and B Merging two sorted sequences, each with n  2 elements and implemented by means of a doubly linked list, takes O(n) time Algorithm merge(A, B) Input sequences A and B with n  2 elements each Output sorted sequence of A  B S  empty sequence while  A.isEmpty()   B.isEmpty() if A.first().element() < B.first().element() S.insertLast(A.remove(A.first())) else S.insertLast(B.remove(B.first())) while  A.isEmpty() S.insertLast(A.remove(A.first())) while  B.isEmpty() S.insertLast(B.remove(B.first())) return S

4. Divide-and-Conquer4 Merge-Sort Tree An execution of merge-sort is depicted by a binary tree each node represents a recursive call of merge-sort and stores  unsorted sequence before the execution and its partition  sorted sequence at the end of the execution the root is the initial call the leaves are calls on subsequences of size 0 or 1 7 2  9 4  2 4 7 9 7  2  2 79  4  4 9 7  72  29  94  4

5. Divide-and-Conquer5 Execution Example Partition 7 2 9 4  3 8 6 1

6. Divide-and-Conquer6 Execution Example (cont.) Recursive call, partition 7 2  9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 2  2 79 4  4 93 8  3 86 1  1 67  72  29  94  43  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

7. Divide-and-Conquer7 Execution Example (cont.) Recursive call, partition 7 2  9 4  2 4 7 93 8 6 1  1 3 8 6 7  2  2 7 9 4  4 93 8  3 86 1  1 6 7  72  29  94  43  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

8. Divide-and-Conquer8 Execution Example (cont.) Recursive call, base case 7 2  9 4  2 4 7 93 8 6 1  1 3 8 6 7  2  2 79 4  4 93 8  3 86 1  1 6 7  77  7 2  29  94  43  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

9. Divide-and-Conquer9 Execution Example (cont.) Recursive call, base case 7 2  9 4  2 4 7 93 8 6 1  1 3 8 6 7  2  2 79 4  4 93 8  3 86 1  1 6 7  77  72  22  29  94  43  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

10. Divide-and-Conquer10 Execution Example (cont.) Merge 7 2  9 4  2 4 7 93 8 6 1  1 3 8 6 7  2  2 7 9 4  4 93 8  3 86 1  1 6 7  77  72  22  29  94  43  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

11. Divide-and-Conquer11 Execution Example (cont.) Recursive call, …, base case, merge 7 2  9 4  2 4 7 93 8 6 1  1 3 8 6 7  2  2 7 9 4  4 9 3 8  3 86 1  1 6 7  77  72  22  23  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9 9  94  4

12. Divide-and-Conquer12 Execution Example (cont.) Merge 7 2  9 4  2 4 7 9 3 8 6 1  1 3 8 6 7  2  2 79 4  4 93 8  3 86 1  1 6 7  77  72  22  29  94  43  38  86  61  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

13. Divide-and-Conquer13 Execution Example (cont.) Recursive call, …, merge, merge 7 2  9 4  2 4 7 9 3 8 6 1  1 3 6 8 7  2  2 79 4  4 93 8  3 86 1  1 6 7  77  72  22  29  94  43  33  38  88  86  66  61  11  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

14. Divide-and-Conquer14 Execution Example (cont.) Merge 7 2  9 4  2 4 7 93 8 6 1  1 3 6 8 7  2  2 79 4  4 93 8  3 86 1  1 6 7  77  72  22  29  94  43  33  38  88  86  66  61  11  1 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

15. Divide-and-Conquer15 Recurrence Equation Analysis The conquer step of merge-sort consists of merging two sorted sequences, each with n  2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b. Likewise, the basis case ( n < 2) will take at b most steps. Therefore, if we let T(n) denote the running time of merge-sort: We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. That is, a solution that has T(n) only on the left-hand side.

16. Divide-and-Conquer16 Iterative Substitution In the iterative substitution, or “plug-and-chug,” technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: Note that base, T(1)=b, case occurs when 2 i =n. That is, i = log n. So, Thus, T(n) is O(n log n).

17. Divide-and-Conquer17 The Recursion Tree Draw the recursion tree for the recurrence relation and look for a pattern: depthT’ssize 01n 12 n2n2 i2i2i n2in2i ……… time bn … Total time = bn + bn log n (last level plus all previous levels)

18. Divide-and-Conquer18 Guess-and-Test Method In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction: Guess: T(n) < cn log n. Wrong: we cannot make this last line be less than cn log n

19. Divide-and-Conquer19 Guess-and-Test Method, Part 2 Recall the recurrence equation: Guess #2: T(n) < cn log 2 n. if c > b. So, T(n) is O(n log 2 n). In general, to use this method, you need to have a good guess and you need to be good at induction proofs.

20. Divide-and-Conquer20 Master Method Many divide-and-conquer recurrence equations have the form: The Master Theorem:

21. Divide-and-Conquer21 Master Method, Example 1 The form: The Master Theorem: Example: Solution: log b a=2, so case 1 says T(n) is Θ(n 2 ).

22. Divide-and-Conquer22 Master Method, Example 2 The form: The Master Theorem: Example: Solution: log b a=1, so case 2 says T(n) is Θ(n log 2 n).

23. Divide-and-Conquer23 Master Method, Example 3 The form: The Master Theorem: Example: Solution: log b a=0, so case 3 says T(n) is Θ(n log n).

24. Divide-and-Conquer24 Master Method, Example 4 The form: The Master Theorem: Example: Solution: log b a=3, so case 1 says T(n) is Θ(n 3 ).

25. Divide-and-Conquer25 Master Method, Example 5 The form: The Master Theorem: Example: Solution: log b a=2, so case 3 says T(n) is Θ(n 3 ).

26. Divide-and-Conquer26 Master Method, Example 6 The form: The Master Theorem: Example: Solution: log b a=0, so case 2 says T(n) is Θ(log n). (binary search)

27. Divide-and-Conquer27 Master Method, Example 7 The form: The Master Theorem: Example: Solution: log b a=1, so case 1 says T(n) is Θ(n). (heap construction)

28. Divide-and-Conquer28 Iterative “Proof” of the Master Theorem Using iterative substitution, let us see if we can find a pattern:

29. Divide-and-Conquer29 Iterative “Proof” of the Master Theorem We then distinguish the three cases as The first term is dominant Each part of the summation is equally dominant The summation is a geometric series

30. Divide-and-Conquer30 Dyn. Prog. vs. Divide and Conquer Divide and conquer: top-down Dynamic programming: bottom-up