Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser.

Similar presentations


Presentation on theme: "Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser."— Presentation transcript:

1 Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser

2 L2.2 Solving recurrences The analysis of integer multiplication from Lecture 1 required us to solve a recurrence. Recurrences are a major tool for analysis of algorithms Today: Learn a few methods. Lecture 3: Divide and Conquer algorithms which are analyzable by recurrences.

3 L2.3 Recall: Integer Multiplication Let X = A B and Y = C D where A,B,C and D are n/2 bit integers Simple Method: XY = (2 n/2 A+B)(2 n/2 C+D) Running Time Recurrence T(n) < 4T(n/2) + 100n How do we solve it?

4 L2.4 Substitution method 1.Guess the form of the solution. 2.Verify by induction. 3.Solve for constants. The most general method: Example: T(n) = 4T(n/2) + 100n [Assume that T(1) =  (1).] Guess O(n 3 ). (Prove O and  separately.) Assume that T(k)  ck 3 for k < n. Prove T(n)  cn 3 by induction.

5 L2.5 Example of substitution desired – residual whenever (c/2)n 3 – 100n  0, for example, if c  200 and n  1. desired residual 3 33 3 3 ) )2/(( )2/( )2/(4 )2/(4)( cn 100n nccn 100nnc nc nTnT      

6 L2.6 Example (continued) We must also handle the initial conditions, that is, ground the induction with base cases. Base: T(n) =  (1) for all n < n 0, where n 0 is a suitable constant. For 1  n < n 0, we have “  (1)”  cn 3, if we pick c big enough. This bound is not tight!

7 L2.7 A tighter upper bound? We shall prove that T(n) = O(n 2 ). Assume that T(k)  ck 2 for k < n: for no choice of c > 0. Lose! )2/(4)( 2 100ncn 100nnTnT   2 cn  

8 L2.8 A tighter upper bound! I DEA : Strengthen the inductive hypothesis. Subtract a low-order term. Inductive hypothesis: T(k)  c 1 k 2 – c 2 k for k < n. if c 2 > 100. Pick c 1 big enough to handle the initial conditions. ) ( 2 ))2/()2/((4 )2/(4)( 2 2 1 22 2 1 2 2 1 2 2 1 ncnc 100n ncncnc ncnc ncnc nTnT     

9 L2.9 Recursion-tree method A recursion tree models the costs (time) of a recursive execution of an algorithm. The recursion tree method is good for generating guesses for the substitution method. The recursion-tree method can be unreliable, just like any method that uses ellipses (…). The recursion-tree method promotes intuition, however.

10 L2.10 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 :

11 L2.11 Example of recursion tree T(n)T(n) Solve T(n) = T(n/4) + T(n/2) + n 2 :

12 L2.12 Example of recursion tree T(n/4) T(n/2) n2n2 Solve T(n) = T(n/4) + T(n/2) + n 2 :

13 L2.13 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : n2n2 (n/4) 2 (n/2) 2 T(n/16)T(n/8) T(n/4)

14 L2.14 Example of recursion tree (n/16) 2 (n/8) 2 (n/4) 2 (n/2) 2  (1) … Solve T(n) = T(n/4) + T(n/2) + n 2 : n2n2

15 L2.15 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/16) 2 (n/8) 2 (n/4) 2 (n/2) 2  (1) … n2n2

16 L2.16 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/16) 2 (n/8) 2 (n/4) 2 (n/2) 2  (1) … n2n2

17 L2.17 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/16) 2 (n/8) 2 (n/4) 2  (1) … n2n2 (n/2) 2 …

18 L2.18 Example of recursion tree Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/16) 2 (n/8) 2 (n/4) 2  (1) … … Total = =  (n 2 ) n2n2 (n/2) 2 geometric series

19 L2.19 Appendix: geometric series for |x| < 1 for x  1 Return to last slide viewed.

20 L2.20 The master method The master method applies to recurrences of the form T(n) = a T(n/b) + f (n), where a  1, b > 1, and f is asymptotically positive.

21 L2.21 f (n/b) Idea of master theorem f (n/b)   (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … #leaves = a h = a log b n = n log b a n log b a   (1)

22 L2.22 Three common cases Compare f (n) with n log b a : 1. f (n) = O(n log b a –  ) for some constant  > 0. f (n) grows polynomially slower than n log b a (by an n  factor). Solution: T(n) =  (n log b a ).

23 L2.23 f (n/b) Idea of master theorem f (n/b)   (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … n log b a   (1) C ASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.  (n log b a )

24 L2.24 Three common cases Compare f (n) with n log b a : 2. f (n) =  (n log b a lg k n) for some constant k  0. f (n) and n log b a grow at similar rates. Solution: T(n) =  (n log b a lg k+1 n).

25 L2.25 f (n/b) Idea of master theorem f (n/b)   (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … n log b a   (1) C ASE 2: (k = 0) The weight is approximately the same on each of the log b n levels.  (n log b a lg n)

26 L2.26 Three common cases (cont.) Compare f (n) with n log b a : 3. f (n) =  (n log b a +  ) for some constant  > 0. f (n) grows polynomially faster than n log b a (by an n  factor), and f (n) satisfies the regularity condition that a f (n/b)  c f (n) for some constant c < 1. Solution: T(n) =  ( f (n) ).

27 L2.27 f (n/b) Idea of master theorem f (n/b)   (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … n log b a   (1) C ASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.  ( f (n))

28 L2.28 Examples Ex. T(n) = 4T(n/2) + n a = 4, b = 2  n log b a = n 2 ; f (n) = n. C ASE 1: f (n) = O(n 2 –  ) for  = 1.  T(n) =  (n 2 ). Ex. T(n) = 4T(n/2) + n 2 a = 4, b = 2  n log b a = n 2 ; f (n) = n 2. C ASE 2: f (n) =  (n 2 lg 0 n), that is, k = 0.  T(n) =  (n 2 lg n).

29 L2.29 Examples Ex. T(n) = 4T(n/2) + n 3 a = 4, b = 2  n log b a = n 2 ; f (n) = n 3. C ASE 3: f (n) =  (n 2 +  ) for  = 1 and 4(cn/2) 3  cn 3 (reg. cond.) for c = 1/2.  T(n) =  (n 3 ). Ex. T(n) = 4T(n/2) + n 2 /lg n a = 4, b = 2  n log b a = n 2 ; f (n) = n 2 /lg n. Master method does not apply. In particular, for every constant  > 0, we have n   (lg n).

30 L2.30 Conclusion Next time: applying the master method. For proof of master theorem, goto section


Download ppt "Introduction to Algorithms 6.046J Lecture 2 Prof. Shafi Goldwasser."

Similar presentations


Ads by Google