1. Chapter 4: Solution of recurrence relationships
Techniques:
Substitution: Guess answer, prove by induction
Tree analysis: Good way to develop a guess
Master theorem: Recipe when T(n) = aT(n\b) + f(n)
with a >1, b >1 and f(n) a specified function
Example: use substitution method to find run time for
merge sort

2. When n is a power of 2
The recurrence for merge sort (worst case) is
T(n) = n=2
T(n) = 2T(n/2) +n n=2k k>1
Proved by induction on integers (HW1)
T(n) = nlgn= 2k lg2k = k2k

3. Worst case merge sort for any n
T(n) = constant n=1
T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1
Look for order of growth instead of analytic solution
We can ignore floor and ceiling and replace Q(n) by
cn with c > 0
Recurrence becomes T(n) = 2T(n/2) + cn
Prove T(n) = Q(nlgn) by substitution method

4. Based on Theorem 3.1, T(n) = Q(nlgn) iff
T(n) = O(nlgn) and T(n) = W(nlgn)
Start by proving T(n) = O(nlgn) is a solution of
T(n) = 2T(n/2) + cn
if -dn + cn = 0; therefore d = c > 0 will work
as the constant in the definition of big O

5. For a more general solution start with the string on inequalities
T(n) < dnlg(n)-dn+cn < dnlg(n),
which says
if T(n) < dnlg(n)-dn+cn
and dnlg(n)-dn+cn < dnlg(n)
then T(n) < dnlg(n),
which is what we need in order to show that T(n)=O(nlg(n)).
But T(n) < dnlg(n)-dn+cn < dnlg(n)
also put a condition on d by the inequality
dnlg(n)-dn+cn < dnlg(n)
Solving this inequality for d yields
d > c > 0,
which says that d = c is OK as the constant in the definition of big O but any d bigger than c is also OK

6. Finish by proving T(n) = W(nlgn) is a solution of T(n) = 2T(n/2) + cn
Assume T(n/2)= W((n/2)lg(n/2))
if -dn + cn = 0; therefore d = c > 0 will work
as the constant in the definition of big W

7. For a more general solution start with the string on inequalities
T(n) > dnlg(n)-dn+cn > dnlg(n),
which says
if T(n) > dnlg(n)-dn+cn
and dnlg(n)-dn+cn > dnlg(n)
then T(n) > dnlg(n),
which is what we need in order to show that T(n)=W(nlg(n)).
But T(n) > dnlg(n)-dn+cn > dnlg(n)
also put a condition on d by the inequality
dnlg(n)-dn+cn > dnlg(n)
Solving this inequality for d yields
0 < d < c,
which says that d = c is OK as the constant in the definition of big W but any d between 0 and c is also OK

8. Sometimes, even with correct guess,
substitution does not work
To prove Big O in these cases, try subtracting
a lower-order term in your assumption
Example: T(n) = 8T(n/2) + Q(n2)
Show T(n) = O(n3)

9. The string of inequalities T(n) < dn3 + cn2 < dn3
cannot be true because cn2 > 0
The substitution T(n/2) = O((n/2)3 will not work in this case
Try “subtract off a lower order term”
assume T(n/2) < d[(n/2)3 – (n/2)2]
if d = c > 0

10. if we assume T(n/2) < d[(n/2)3 – (n/2)2]
then we most show T(n) < d[n3-n2]
< d[n3-n2] implies
T(n) < d[n3-n2] and 0 < c < d
(i.e. any d > c satisfies the definition of big O)
if d = c > 0

11. Substitution T(n/2) = W((n/2)3) works for proving T(n) = W(n3)
is a solution of T(n) = 8T(n/2) + cn2 with c > 0
Assuming T(n/2) = W((n/2)3) means there exist d>0 such that
T(n/2) > d(n/2)3 which implies
T(n) > 8d(n/2)3 + cn2
The string of inequalities T(n) > dn3 + cn2 > dn3 can be true because cn2 > 0,
which means that any d > 0 is OK as the constant in the definition of big W; hence T(n) = W(n3)

12. Sometimes a change of variable helps
Example: T(n) = 2T(n1/2) + lg(n)

13. Change of variable in substitution method

14. Quiz Friday,
Evaluate and bound sums
prove formulas by induction on integers
convert one sum into another
bound by induction
bound every term
bound by integration

15. CptS 450 Spring 2015
[All problems are from Cormen et al, 3rd Edition]
Homework Assignment 4: due 2/11/15
1. ex on page 93 (tree analysis and substitution)
2. problem 4-3f on page 108 by tree analysis and substitution

16. Solution of recurrence by tree analysis

17. Tree analysis
At each node write overhead per recurrence
usually a function of level index
Cost of leaves = number of leaves
Calculate istop
Sum level cost for i = 0 to istop -1
Add cost of leaves
If tree analysis is ambiguous, verify by substitution
Example: T(n) = 3T(|_n/4_|) + Q(n2) ~ 3T(n/4) + cn2

18. T(n)=3T(n/4)+cn2 cost of level i
istop = ? How many leaves?

19. As indicated in previous slide,
levels are indexed from zero
ISTOP is the index of leaves
At ISTOP divide and conquer
has bottomed out with subgroups
of size n0
Could evaluate cost of level
using finite geometric sum
series
Bounding by infinite series easier

20. Test guess from tree analysis by substitution

21. Test guess from tree analysis by substitution
Guess is upper bound only
Did not evaluate sum over levels
< dn2
solving right-most inequality for d yields d > (16/13)c > 0
for all n > 1
there exists d > 0 such that 0 < T(n) < dn2 all n > 1
hence, T(n) = O(n2)
= 1

22. Binary tree with complicated level cost
T(n) = 2T(n/2 +17) +n
Expect T(n) = nlgn Why?

23. Note: sum cannot be bounded by
geometric series. Why?

24. HW 4 due date changed
Homework Assignment 4: due 2/13/15
1. ex on page 93 (tree analysis and substitution)
2. problem 4-3f on page 108 by tree analysis and substitution
Quiz Friday 2/13
Substitution method
set up (remove floors, ceiling, asymptotic notation)
assumption
implementation of assumption
constants and n0

25. Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

26. Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

27. Imbalanced tree analysis continued
Test guess by substitution
< dnlgn
solve
< 0 for d
d > c/(lg3-2/3) > 0 =
all n > 0

28. Skinny Trees

29. Skinny tree
leaf = no2
no2
no2
See text p1147
no2
+ Q(n3)

30. substitution < cn3 T(n) < cn3 - 6cn2 + 12cn -8c + n2 < cn3
Solve - 6cn2 + 12cn -8c + n2 < 0 for c
In this case, -c6n2+12cn-8c+n2 = 0 leads to c = f(n)
n2 (1 – 6c) + 12cn - 8c < 0
Look for a combination of c and n0 that works in definition of big O
n2 (1 – 6c) will be the dominate term at large n
Try c = 1 and see how large n must be
-5n2 + 12n - 8 < 0 true for all n>1
c = n0 = 1 work in definition of big O
< cn3
= 0

31. T(n) > cn3 - 6cn2 + 12cn -8c + n2 > cn3
Solve - 6cn2 + 12cn -8c + n2 > 0 for c
Look for a combination of c and n0 that works in definition of big W
c = 1/6 eliminates n2 term
2n – 4/3 > 0 true for all n>1
c = 1/6 and n0 = 1 work in definition of big W

32. Master Theorem: (statement on p94)
“cook-book” method to solve T(n) = aT(n/b) + f(n) when
a > 1, b > 1, and f(n) asymptotically positive
Ignore floors and ceiling
do not effect validity (section p103)
At large n, compare f(n) to
Case 1:
is polynomially larger than f(n) at large n
Conclusion: T(n) = Q(
We can subtract e > 0 from logb(a) and
is still an
upper bound
on f(n)
There exist e > 0 such that f(n) = O(

33. is asymptotically the same as f(n)
Case 2:
is asymptotically the same as f(n)
Conclusion: )
f(n) = Q(
lgn) = Q(f(n)lgn)
T(n) = Q(
Case 3:
f(n) is polynomially larger than
There exist e > 0 such that f(n) = W(
We can add e > 0 from logb(a) and
is still a lower
bound on f(n)
f(n) is “regular” af(n/b) < cf(n) for some c < 1 and n > n0
Conclusion: T(n) = Q(f(n))

34. Cpt S 450 Spring 2015
[All problems are from Cormen et al, 3rd Edition]
Homework Assignment 5: due 2/18/15
Problems 4-1a, c and e on page 107 by master method

35. When is Master Theorem not applicable
1. T(n) does not have the correct form
2. f(n) falls between cases 1 and 2
(cannot find e > 0 required by case 1)
3. f(n) falls between cases 2 and 3
(cannot find e > 0 required by case 3)
4. Regularity condition fails in case 3

36. Prove that, in case 3, polynomial f(n) is regular

37. Prove that, in case 3, polynomial f(n) is regular
Given f(n) = nk and f(n) = W( )

38. Quiz Friday
Tree analysis
size of tree (istop)
cost of leaves (number of leaves)
cost of ith level
total cost of levels
total cost
order of growth

39. Proof of Master Theorem
MT applies to T(n) = aT(n/b) + f(n) with a>1 and b>1
Analyze master recurrence for n = bk
(no floors or ceilings)
1. tree analysis → solution as a sum
2. bound sum → 3 cases
B. Extend solution to any n > 0
Apply techniques related to floors and ceilings

40. Tree analysis master theorem
T(n) = aT(n/b) + f(n) bi bi

41. T(n) = aT(n/b) + f(n) bi bi Case 1: cost of leaves dominates
Case 2: cost evenly distributed over levels and leaves
Case 3: cost of root dominates: T(n) = Q(f(n))

42. T(n) = aT(n/b) + f(n)
When n is not a power of b recurrence becomes
T(n) = constant n=1
T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1
Section show that floors and ceiling do not
affect the tree analysis

43. Quiz Friday,
Master Theorem
Which case applies
Relationship between f(n) and nlogb(a) in asymptotic
notation
Acceptable values of e, cases 1 and 3
Regularity of f(n), case 3

44. Iteration on recurrence T(n)=4T(n/2)+cn
T(n) = cn + 4T(n/2)
T(n) = cn + 4(cn/2 + 4T(n/4))
T(n) = cn +2cn +16(cn/4 + 4T(n/8))
T(n) = cn +2cn + 4cn +64(cn/8+4T(n/16))
T(n) = cn +2cn + 4cn + 8cn +256(cn/16 + 4T(n/32))
Level = …
Cost of level k is 2kcn
T(n)=Sk=0 to istop 2kcn=cn(2lg(n)+1-1)/(2-1)=cn(2nlg(2)-1)
Compare to solution by master’s theorem

45. More iteration examples
T(n)=T(n-2)+n2
T(n)=T(n/2)+T(n/4)+T(n/8)