1. Solving recurrence equations
5 techniques for solving recurrence equations:
Recursion tree
Iteration method
Induction (Guess and Test)
Master Theorem (master method)
Characteristic equations
We concentrate on the recursion tree method (others in the book)
5/13/2019
cutler: divide & conquer

2. Deriving the count using the recursion tree method
Recursion trees provide a convenient way to represent the unrolling of recursive algorithm
It is not a formal proof but a good technique to compute the count.
Once the tree is generated, each node contains its “non recursive number of operations” t(n) or DirectSolutionCount
The count is derived by summing the “non recursive number of operations” of all the nodes in the tree
For convenience we usually compute the sum for all nodes at each given depth, and then sum these sums over all depths.
5/13/2019
cutler: divide & conquer

3. Building the recursion tree
The initial recursion tree has a single node containing two fields:
The recursive call, (for example Factorial(n)) and
the corresponding count T(n) .
The tree is generated by:
unrolling the recursion of the node depth 0,
then unrolling the recursion for the nodes at depth 1, etc.
The recursion is unrolled as long as the size of the recursive call is greater than DirectSolutionSize
5/13/2019
cutler: divide & conquer

4. Building the recursion tree
When the “recursion is unrolled”, each current leaf node is substituted by a subtree containing a root and a child for each recursive call done by the algorithm.
The root of the subtree contains the recursive call, and the corresponding “non recursive count”.
Each child node contains a recursive call, and its corresponding count.
The unrolling continues, until the “size” in the recursive call is DirectSolutionSize
Nodes with a call of DirectSolutionSize, are not “unrolled”, and their count is replaced by DirectSolutionCount
5/13/2019
cutler: divide & conquer

5. Example: recursive factorial
Factorial(n)
T(n)
Initially, the recursive tree is a node containing the call to Factorial(n), and count T(n).
When we unroll the computation this node is replaced with a subtree containing a root and one child:
The root of the subtree contains the call to Factorial(n) , and the “non recursive count” for this call t(n)= (1).
The child node contains the recursive call to Factorial(n-1), and the count for this call, T(n-1).
5/13/2019
cutler: divide & conquer

6. After the first unrolling
depth nd T(n)
Factorial(n)
t(n)= (1)
(1)
T(n-1)
Factorial(n-1)
T(n-1)
nd denotes the number of nodes at that depth
5/13/2019
cutler: divide & conquer

7. After the second unrolling
depth nd T(n)
Factorial(n)
t(n)= (1)
(1)
Factorial(n-1)
t(n-1)= (1)
(1)
Factorial(n-2)
T(n-2)
T(n-2)
5/13/2019
cutler: divide & conquer

8. After the third unrolling
depth nd T(n)
Factorial(n)
t(n)= (1)
(1)
Factorial(n-1)
t(n-1)= (1)
(1)
Factorial(n-2)
t(n-2)= (1)
(1)
Factorial(n-3)
T(n-3)
T(n-3)
For Factorial DirectSolutionSize = 1 and DirectSolutionCount= (1)
5/13/2019
cutler: divide & conquer

9. cutler: divide & conquer
The recursion tree
depth nd T(n)
Factorial(n)
t(n)= (1)
(1)
Factorial(n-1)
t(n-1)= (1)
(1)
Factorial(n-2)
t(n-2)= (1)
(1)
Factorial(n-3)
t(n-3)= (1)
(1)
Factorial(1)
T(1)=
(1)
n T(1)= (1)
The sum T(n)=n* (1) = (n)
5/13/2019
cutler: divide & conquer

10. cutler: divide & conquer
Divide and Conquer
Basic idea: divide a problem into smaller portions, solve the smaller portions and combine the results.
Name some algorithms you already know that employ this technique.
D&C is a top down approach. Often, you use recursion to implement D&C algorithms.
The following is an “outline” of a divide and conquer algorithm
5/13/2019
cutler: divide & conquer

11. cutler: divide & conquer
Divide and Conquer
procedure Solution(I);
begin
if size(I) <=smallSize then {calculate solution}
return(DirectSolution(I)) {use direct solution}
else {decompose, solve each and combine}
Decompose(I, I1,...,Ik);
for i:=1 to k do
S(i):=Solution(Ii); {solve a smaller problem}
end {for}
return(Combine(S1,...,Sk)); {combine solutions}
end {Solution}
5/13/2019
cutler: divide & conquer

12. cutler: divide & conquer
Divide and Conquer
Let size(I) = n
DirectSolutionCount = DS(n)
t(n) = D(n) + C(n) where:
D(n) = count for dividing problem into subproblems
C(n) = count for combining solutions
5/13/2019
cutler: divide & conquer

13. cutler: divide & conquer
Divide and Conquer
Main advantages
Simple code
Often efficient algorithms
Suitable for parallel computation
5/13/2019
cutler: divide & conquer

14. cutler: divide & conquer
Binary search
Assumption - the list S[low…high] is sorted, and x is the search key
If the search key x is in the list, x==S[i], and the index i is returned.
If x is not in the list a NoSuchKey is returned
Book’s version: returns an index i where x would be inserted into the sorted list
5/13/2019
cutler: divide & conquer

15. cutler: divide & conquer
Binary search
The problem is divided into 3 subproblems: x=S[mid], xÎ S[low,..,mid-1], xÎ S[mid+1,..,high]
The first case x=S[mid]) is easily solved
The other cases xÎ S[low,..,mid-1], or xÎ S[mid+1,..,high] require a recursive call
When the array is empty the search terminates with a “non-index value”
5/13/2019
cutler: divide & conquer

16. cutler: divide & conquer
BinarySearch(S, k, low, high)
if low > high then
return NoSuchKey
else
mid  floor ((low+high)/2)
if (k == S[mid])
return mid
else if (k < S[mid]) then
return BinarySearch(S, k, low, mid-1)
return BinarySearch(S, k, mid+1, high)
5/13/2019
cutler: divide & conquer

17. cutler: divide & conquer
Worst case analysis
A worst input (what is it?) causes the algorithm to keep searching until low>high
We count number of comparisons of a list element with x per recursive call.
Assume 2k  n < 2k+1 k = ëlg nû
T (n) - worst case number of comparisons for the call to BS(n)
5/13/2019
cutler: divide & conquer

18. Recursion tree for BinarySearch (BS)
BS(n)
T(n)
Initially, the recursive tree is a node containing the call to BS(n), and total amount of work in the worst case, T(n).
When we unroll the computation this node is replaced with a subtree containing a root and one child:
The root of the subtree contains the call to BS(n) , and the “nonrecursive work” for this call t(n).
The child node contains the recursive call to BS(n/2), and the total amount of work in the worst case for this call, T(n/2).
5/13/2019
cutler: divide & conquer

19. cutler: divide & conquer
After first unrolling
depth nd T(n)
BS(n)
t(n)=1
BS(n/2)
T(n/2)
T(n/2)
5/13/2019
cutler: divide & conquer

20. After second unrolling
depth nd T(n)
BS(n)
t(n)=1
BS(n/2)
t(n/2)=1
BS(n/4)
T(n/4)
T(n/4)
5/13/2019
cutler: divide & conquer

21. cutler: divide & conquer
After third unrolling
depth nd T(n)
BS(n)
t(n)=1
BS(n/2)
t(n/2)=1
BS(n/4)
t(n/4)=1
BS(n/8)
T(n/8)
T(n/8)
For BinarySearch DirectSolutionSize =0, 1 and DirectSolutionCount=0 for 0, and 1 for 1
5/13/2019
cutler: divide & conquer

22. Terminating the unrolling
Let 2k n < 2k+1
k= lg n
When a node has a call to BS(n/2k), (or to BS(n/2k+1)):
The size of the list is DirectSolutionSize since   n/2k  = 1, (or  n/2k+1  = 0)
In this case the unrolling terminates, and the node is a leaf containing DirectSolutionCount (DFC) = 1, (or 0)
5/13/2019
cutler: divide & conquer

23. cutler: divide & conquer
The recursion tree
depth nd T(n)
BS(n)
t(n)=1
BS(n/2)
t(n/2)=1
BS(n/4)
t(n/4)=1
BS(n/2k)
DFC=1 k
T(n)=k+1=lgn +1
5/13/2019
cutler: divide & conquer

24. cutler: divide & conquer
Merge Sort
Input: S of size n.
Output: a permutation of S, such that if i > j then S[ i ]  S[ j ]
Divide: If S has at least 2 elements divide into S1 and S2. S1 contains the the first  n/2  elements of S. S2 has the last  n/2 elements of S.
Recurs: Recursively sort S1 , and S2.
Conquer: Merge sorted S1 and S2 into S.
5/13/2019
cutler: divide & conquer

25. cutler: divide & conquer
Merge Sort:
Input: S 1 if (n  2) 2 moveFirst( S, S1, (n+1)/2))// divide moveSecond( S, S2, n/2) // divide 4 Sort(S1) // recurs Sort(S2) // recurs 6 Merge( S1, S2, S ) // conquer
5/13/2019
cutler: divide & conquer

26. Merge( S1 , S2, S ): Pseudocode
Input: S1 , S2, sorted in nondecreasing order
Output S is a sorted sequence. 1. while (S1 is Not Empty && S2 is Not Empty) 2. if (S1.first()  S2.first() ) remove first element of S1 and move it to the end of S else remove first element of S2 and move it to the end of S 5. move the remaining elements of the non-empty sequence (S1 or S2 ) to S
5/13/2019
cutler: divide & conquer

27. Recurrence Equation for MergeSort.
The implementation of the moves costs Q(n) and the merge costs Q(n). So the total count for dividing and merging is Q(n).
The recurrence relation for the run time of MergeSort is T(n) = T( n/2 ) + T( n/2 ) + Q(n) . T(1) = Q(1)
5/13/2019
cutler: divide & conquer

28. Recursion tree for MergeSort (MS)
MS(n)
T(n)
Initially, the recursive tree is a node containing the call to MS(n), and total amount of work in the worst case, T(n).
When we unroll the computation this node is replaced with a subtree:
The root of the subtree contains the call to MS(n) , and the “nonrecursive work” for this call t(n).
The two children contain a recursive call to MS(n/2), and the total amount of work in the worst case for this call, T(n/2).
5/13/2019
cutler: divide & conquer

29. After first unrolling of mergeSort
MS(n)
depth nd T(n)
t(n)=(n)
(n)
MS(n/2)
T(n/2)
MS(n/2)
T(n/2)
2T(n/2)
5/13/2019
cutler: divide & conquer

30. After second unrolling
depth nd T(n)
MS(n)
(n)
(n)
MS(n/2)
(n/2)
MS(n/2)
(n/2)
(n) +
MS(n/4)
T(n/4)
MS(n/4)
T(n/4)
MS(n/4)
T(n/4)
4T(n/4)
5/13/2019
cutler: divide & conquer

31. cutler: divide & conquer
After third unrolling
depth nd T(n)
MS(n)
(n)
(n)
MS(n/2)
(n/2)
MS(n/2)
(n/2)
(n) +
MS(n/4)
(n/4)
MS(n/4)
(n/4)
MS(n/4)
(n/4)
MS(n/4)
(n/4)
(n) + + +
MS(n/8)
T(n/8)
MS(n/8)
T(n/8)
MS(n/8)
T(n/8)
8T(n/8)
5/13/2019
cutler: divide & conquer

32. Terminating the unrolling
For simplicity let n = 2k
lg n = k
When a node has a call to MS(n/2k):
The size of the list to merge sort is DirectSolutionSize since n/2k = 1
In this case the unrolling terminates, and the node is a leaf containing DirectSolutionCount = (1)
5/13/2019
cutler: divide & conquer

33. The recursion tree (excluding the calls) d nd T(n)
Q (n/20)
Q (n/ 21)
Q (n / 21)
Q (n/ 22)
Q (n / 22)
Q (n / 8)
Q (n/ 8)
Q (n / 23)
Q (1)
Q (n)
Q (n)
Q (n)
Q (n)
k 2k Q (n)
T(n)= (k+1) Q (n) = Q( n lg n)
5/13/2019
cutler: divide & conquer

34. cutler: divide & conquer
The sum for each depth
d=0 nd=1: 21 22 .
Q (n )
´ Q n æ è ç ö ø ÷
= Q (n)
d=1 nd=2:
d=2 nd=22:
d=k nd=2k:
5/13/2019
cutler: divide & conquer