1. Trevor Brown trevor.brown@uwaterloo.ca
CS 341: Algorithms
Trevor Brown

2. This time Divide and conquer paradigm (intro to…) Merge sort
Recurrence relations
Tree recursion method: merge sort as an example
Guess and check method
[time permitting] Master theorem

3. Divide and conquer
A useful design strategy / paradigm

4. Divide-and-Conquer Design Strategy
divide: Given a problem instance I, construct one or more smaller problem instances I1, …, Ia
These are called subproblems
Usually, want subproblems to be small compared to the size of I (e.g., half the size)
conquer: For 1 ≤ j ≤ a, solve instance Ij recursively, obtaining solutions S1, …, Sa
combine: Given solutions S1, …, Sa, use an appropriate combining function to find the solution S to the problem instance I
i.e., S = Combine(S1, …, Sa).

5. Example: Design of Mergesort

6. divide
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12

7. Merge: Conquer and combine1 4 5 7 8 10 11 12 13 14 19 21 31 96 98
105 1 7 8 11 13 14 19
105 4 5 10 12 21 31 96 98 7 8 13
105 1 11 14 19 4 10 96 98 5 12 21 31 7
105 8 13 1 14 11 19 4 10 16 98 5 31 12 21
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12

8. Merge simulation 1 L R 5 31 12 21 O 5 12 5 12 21 31 5 5 12 21

9. Another example merge step1 4 5 7 8 10 11 12 13 14 19 21 31 96 98
105 1 7 8 11 13 14 19
105 4 5 10 12 21 31 96 98 7 8 13
105 1 11 14 19 4 10 96 98 5 12 21 31 7
105 8 13 1 14 11 19 4 10 16 98 5 31 12 21
105 7 13 8 14 1 19 11 4 10 98 16 31 5 21 12

10. Merge simulation 2 R L 4 10 96 98 5 12 21 31 O 4 5 10 12 21 31 96 98 4 5 10 12 21 31 96 4 5 10 4 5 4 5 10 12 4 5 10 12 21 4 5 10 12 21 31 4

11. Pseudocode for mergesort

12. Pseudocode for merge
While both arrays still contain elements to be merged
Handle case where
the right array empties first (and left has some elements)
Handle case where
the left array empties first

13. Analysis of mergesort
MergeSort(A, n) takes O(n) time, plus the time for its two recursive calls!
O(1)
O(1)
O(1)
O(n)
O(n)
???
???
O(n)

14. T(n) is a function of T(…). T is a recurrence relation
More formally
T(n) is a function of T(…). T is a recurrence relation
How can we compute T(n)?

15. Analysis of Mergesort
msort(n)
cn +
= cn +
We use the recursion tree method
This is informal! We ignore floor/ceil by assuming n=2k, and skip lots of math
msort(n/2)
2(cn/2) +
= cn +
msort(n/2)
4(cn/4) +
= cn +
msort(n/4)
msort(n/4)
msort(n/4)
msort(n/4) …. …. …. …. …. ….
msort(1)
msort(1)
msort(1)
msort(1)
n(c)
= cn
Total = cn*#levels
Total = cn log2(n)
MergeSort takes O(n log n) time on an input of size n.

16. Recurrence relations
Analysis tool for recursive algorithms

17. Recurrence Relations

18. Guess-and-check Method
Suppose we have the following recurrence
𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5
Guess the form of the solution any way you like
My approach
Recursively substitute the formula into itself
Try to identify patterns to guess the final closed form
Check that the guess was correct

19. Worked example Recurrence: 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5
𝑇 𝑛 = 𝑇 𝑛−2 +6 𝑛−1 −5 +6𝑛−5 (substitute)
=𝑇 𝑛−2 +6𝑛−6−5+6𝑛−5
=𝑇 𝑛−2 +2 6𝑛−5 −6
= 𝑇 𝑛−3 +6 𝑛−2 −5 +2 6𝑛−5 −6 (substitute)
=𝑇 𝑛−3 +6𝑛−2 6 −5+2 6𝑛−5 −6
=𝑇 𝑛−3 +3 6𝑛−5 −6(1+2)
… identify patterns and guess what happens in the limit
=𝑇 𝑛−𝑛 +𝑛 6𝑛−5 − …+ 𝑛−1 =𝑔𝑢𝑒𝑠𝑠(𝑛)
Compare: new terms? +(6n-5)
Compare: new terms? +(6n-5) (6)

20. Recurrence: 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5
g𝑢𝑒𝑠𝑠 𝑛 =𝑇 𝑛−𝑛 +𝑛 6𝑛−5 − …+ 𝑛−1
=𝑇 0 +6 𝑛 2 −5𝑛−6 𝑖=1 𝑛−1 𝑖 (simplify)
=4+6 𝑛 2 −5𝑛−6𝑛(𝑛−1)/2
=4+6 𝑛 2 −5𝑛−3 𝑛 2 +3𝑛
=3 𝑛 2 −2𝑛+4
Are we done?
No! This is a guess.
Must prove it is correct using induction

21. Proof Recurrence: 𝑇 0 =4 ; 𝑇 𝑛 =𝑇 𝑛−1 +6𝑛−5
Our guess: 𝑔𝑢𝑒𝑠𝑠 𝑛 =3 𝑛 2 −2𝑛+4
Want to prove: 𝑔𝑢𝑒𝑠𝑠 𝑛 =𝑇 𝑛 for all 𝑛
Base case: guess 0 = −2 0 +4=𝑇 0
Inductive case: assume 𝑔𝑢𝑒𝑠𝑠 n =𝑇 𝑛 , and show 𝑔𝑢𝑒𝑠𝑠 n+1 =𝑇 𝑛+1 .
𝑇 𝑛+1 =𝑇 𝑛 +6(𝑛+1)−5 (by definition)
=𝑔𝑢𝑒𝑠𝑠 𝑛 +6 𝑛+1 −5 (by inductive hypothesis)
= 3 𝑛 2 −2𝑛+4 +6 𝑛+1 −5=3 𝑛 2 +4𝑛+5
𝑔𝑢𝑒𝑠𝑠 𝑛+1 =3 𝑛+1 2 −2 𝑛+1 +4 (by definition)
=3 𝑛 2 +2𝑛+1 −2𝑛−2+4=3 𝑛 2 +4𝑛+5=𝑇 𝑛+1 . QED.

22. Another approach
Instead of substituting and carefully identifying the pattern, and computing exact constants in our quadratic function, you might simply guess that the result is a quadratic function
𝒂 𝒏 𝟐 +𝒃𝒏+𝒄 for some unknown constants a, b, c
And then carry the unknown constants into the proof!
In the inductive step, you can determine what the constants must be, for the proof to work out

23. Recursion tree method formalized
Sample recurrence for two recursive calls on problem size n/2
Step 1
Step 2
Step 3
Step 4

24. Master theorem for recurrences
Provides a formula for solving many recurrence relations
We start with a simplified version

25. Proof of simplified master theorem
Must sum the values over all levels!

26. Summing over all levels

27. The three cases for 𝑟

28. Math Details for each case
Not covering in detail! Included just in case you are curious, and for your notes.

29. Worked Examples Recall: simplified master theorem
b=2;
y=1;
x=1
Recall: simplified master theorem
Θ 𝑛 𝑥 log 𝑛
=Θ 𝑛 log 𝑛
a=3;
b=2;
y=1;
x=log2 3
Θ 𝑛 𝑥
=Θ 𝑛 log 2 3
a=4;
b=2;
y=1;
x=log2 4
Θ 𝑛 𝑥
=Θ 𝑛 2
Questions: a=? b=? y=? x=?
which Θ function?
a=2;
b=2;
y=3/2;
x=1
Θ 𝑛 𝑦
=Θ 𝑛 3/2

30. General master theorem
Example recurrence:
Arbitrary function of 𝒏
(not just 𝑐 𝑛 𝑦 )
Must reason about relationship between 𝑓(𝑛) and 𝑛 𝑥

31. Revisiting the recursion tree method
Some recurrences with complex f(n) functions (such as f(n) = log n) can still be solved “by hand”
Example: Let n= 2 j ; 𝑇 1 =1 ; 𝑇 𝑛 =2𝑇 𝑛 2 +𝑛 𝑙𝑜𝑔 𝑛
Must sum the values over all levels!

32. Revisiting the recursion tree method
Recall: n= 2 j ; 𝑇 1 =1 ; 𝑇 𝑛 =2𝑇 𝑛 2 +𝑛 𝑙𝑜𝑔 𝑛

33. Next time More divide and conquer problems Non-dominated points
Multi-precision multiplication (very likely)
Selection (somewhat likely)
Closest pair (unlikely)