Chapter 4: Solution of recurrence relationships Techniques: Substitution: proof by induction Tree analysis: graphical representation Master theorem: Recipe when T(n) = aT(n\b) + f(n) with a >1, b >1 and f(n) a specified function

The recurrence for merge sort (worst case) is T(n) = 2 n=2 T(n) = 2T(n/2) +n n=2 k k>1 Proved by induction on integers that T(n) = nlgn = 2 k lg2 k = k2 k Substitution method was used in HW2 to find worst-case run time for merge sort when n is even

Worst case of merge sort for any n T(n) = constant if n=1 T(n) = T(floor(n/2)) + T(ceiling(n/2)) +  (n)if n>1 Look for order of growth instead of analytic solution We can ignore floor and ceiling and replace  (n) by dn with d > 0 Recurrence becomes T(n) = 2T(n/2) + dn Prove T(n) =  (nlgn) by the substitution method Show the constraints on c and n.

Proof that T(n) =  (nlgn) is an asymptotic solution of T(n) = 2T(n/2) +  (n) is sufficient for worst case analysis of merge sort.

To prove that merge sort is asymptotically optimal we need to prove that  (nlgn) is also an asymptotic solution of T(n) = 2T(n/2) +  (n) This proof is part of assignment #7

Assignment 7: due 2/10/16 Prove by substitution that T(n)=2T(n/2)+  (n) has T(n)=  (nlgn) as an asymptotic solution. Show constraints on c and n.

Quiz delayed until Friday I require “structured” solutions to Induction on integers Solution of recurrence relations Tree analysis Structured solution means the steps that I require are identified in your solution. I will not give credit on homework to “unstructured” solutions. On tests, partial credit is given to individual required steps.

Required identified steps in Induction on integers Base case Setup equation for stated technique for example “if S(n-1) then S(n) Inductive hypothesis Application of inductive hypothesis all algebra on the RHS of equations

Required identified steps in solution of recurrence relations by substitution Setup equation remove flour, ceiling and asymptotic notation Inductive hypothesis assumption about T on RHS of recurrence Consequence of inductive hypothesis form of T on RHS to be used in substitution Application of inductive hypothesis solution to a string of inequalities Constraints on c and n 0

Required identified steps in solution of recurrence relations tree analysis Draw sufficient branches to determine cost of level i Calculate “istop” with allowance for leaves >1 Cost of leaves as constant times number of leaves Cost of level i Total cost of levels Solution as total cost (leaves + levels) Solution in asymptotic notation for dominate term

Sometimes, even with a correct guess of the solution, substitution does not work. Example: T(n) = 8T(n/2) +  (n 2 ) Prove T(n) =  O (n 3 )

if d = c > 0 When a substitution proof for Big O fails, try “subtract off a lower order term” In the example, prove T(n) = 8T(n/2) +  (n 2 ) is T(n) =  O(n 3 ) “subtract off a lower order term” means assume T(n/2) = O((n/2) 3 – (n/2) 2 ) Consistent with T(n/2) = O((n/2) 3 ) but more useful When we assume T(n/2) = O((n/2) 3 – (n/2) 2 ), we must prove T(n) = O(n 3 -n 2 ) Again, consistent with T(n) = O(n 3 )

Failure of Big O does not imply fail of Big  More Assignment 7: due 2/10/16 Prove by substitution that T(n)=8T(n/2)+  (n 2 ) has T(n) =  (n 3 ) as an asymptotic solution. Prove by substitution that T(n) = 3T(n/4)+  (n 2 ) has T(n) =  (n 2 ) as an asymptotic solution. Show constraints on c and n.

Sometimes a change of variable helps Example: T(n) = 2T(n 1/2 ) + lg(n)

Change of variable in substitution method

Solution of recurrence by tree analysis Substitution method requires guess of the solution Tree analysis does not require a guess Tree analysis may give approximate solution only Verify result by substitution method

Tree analysis At each node write overhead per recurrence usually a function of level index Cost of leaves = number of leaves Calculate istop Sum level cost for i = 0 to istop -1 Add cost of leaves If tree analysis is ambiguous, verify by substitution Example: T(n) = 3T(|_n/4_|) +  (n 2 ) ~ 3T(n/4) + cn 2

T(n)=3T(n/4)+cn 2 cost of level i istop = ? How many leaves?

As indicated in previous slide, levels are indexed from zero ISTOP is the index of leaves At ISTOP divide and conquer has bottomed out with subgroups of size n 0 Could evaluate cost of level using finite geometric sum Bounding by infinite series easier but result requires testing by the substitution method series

Test guess T(n) = O(n 2 ) from tree analysis by substitution Part of assignment #7

CptS 350 Spring 2016 [All problems are from Cormen et al, 3 rd Edition] Homework Assignment 8: due 2/12/16 ex on page 93 (tree analysis and substitution) Remember! Tight bound means 

Binary tree with complicated level cost T(n) = 2T(n/2 +17) +n Expect T(n) = nlgn Why?

Note: sum cannot be bounded by geometric series. Why?

Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

Imbalanced tree analysis continued Estimates of cost of leaves and cost of levels are super-linear Which dominates at large n? < 0 for d = all n > 0 < dnlgn solve d > c/(lg3-2/3) > 0 Test guess by substitution

Imbalanced tree analysis continued < 0 for d = all n > 0 < dnlgn solve d > c/(lg3-2/3) > 0 Test guess by substitution

CptS 350 Spring 2016 [All problems are from Cormen et al, 3 rd Edition] Homework Assignment 9: due 2/24/16 1.Show by substitution that T(n)=T(floor(n/3))+T(ceiling(2n/3)) +  (n) has asymptotic solution T(n)=  (nlgn) 2. problem 4-3f on page 108 by tree analysis and substitution method for tight bounds.

Skinny Trees

Skinny tree leaf = n o 2 no2no2 no2no2 no2no2 +  (n 3 ) See text p1147

Solve by substitution = 0 < cn 3 T(n) < cn 3 - 6cn cn -8c + n 2 < cn 3 Solve - 6cn cn -8c + n 2 < 0 for c In this case, -c6n 2 +12cn-8c+n 2 = 0 leads to c = f(n) n 2 (1 – 6c) + 12cn - 8c < 0 n 2 (1 – 6c) will be the dominate term at large n (1-6c)<0 0<c<1/6) Try c = 1 and see how large n must be -5n n c = n 0 = 1 work in definition of big O ((n-2) 3 )

CptS 350 Spring 2016 [All problems are from Cormen et al, 3 rd Edition] Homework Assignment 10: due 2/26/16 Show by substitution that T(n)=T(n-2)+  (n 2 ) has asymptotic solution T(n)=  (n 3 )

Master Theorem: (statement on p94) “cook-book” method to solve T(n) = aT(n/b) + f(n) when a > 1, b > 1, and f(n) asymptotically positive Ignore floors and ceiling do not effect validity (section p103) At large n, compare f(n) to Case 1:is polynomially larger than f(n) at large n Conclusion: T(n) =  ( We can subtract  > 0 from log b (a) and is still an upper bound on f(n) There exist  > 0 such that f(n) = O(

Case 2:is asymptotically the same as f(n) Conclusion: ) f(n) =  ( lgn) =  (f(n)lgn) T(n) =  ( Case 3:f(n) is polynomially larger than There exist  > 0 such that f(n) =  ( is still a lower bound on f(n) f(n) is “regular” af(n/b) n 0 Conclusion: T(n) =  (f(n)) We can add  > 0 from log b (a) and

Structured solutions to recursion relations by Master Theorem 1. Which case applies 2. Relationship between f(n) and n logb(a) in asymptotic notation 3. Solution for T(n) 4. Acceptable values of , cases 1 and 3 5. Regularity of f(n), case 3

Cpt S 350 Spring 2016 [All problems are from Cormen et al, 3rd Edition] Homework Assignment 11: due 3/4/16 Problems 4-1a, c and e on page 107 by master method

When is Master Theorem not applicable 1. T(n) does not have the correct form 2. f(n) falls between cases 1 and 2 (cannot find  > 0 required by case 1) 3. f(n) falls between cases 2 and 3 (cannot find  > 0 required by case 3) 4. Regularity condition fails in case 3

Prove that, in case 3, polynomial f(n) is regular

Given f(n) = n k and f(n) =  ( ) Prove that, in case 3, polynomial f(n) is regular