Download presentation

Presentation is loading. Please wait.

1
**Introduction to Algorithms**

LECTURE2 Asymptotic Notation •O-, Ω-, and Θ-notation Recurrences •Substitution method •Iterating the recurrence •Recursion tree •Master method September 12, 2005 Copyright© Erik D. Demaine and Charles E. Leiserson

2
**Asymptotic notation O-notation (upper bounds):**

We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

3
**EXAMPLE: 2n2= O(n3) (c= 1, n0= 2)**

Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. EXAMPLE: 2n2= O(n3) (c= 1, n0= 2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

4
**EXAMPLE: 2n2= O(n3) (c= 1, n0= 2)**

Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. EXAMPLE: 2n2= O(n3) (c= 1, n0= 2) functions, Not values September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

5
**EXAMPLE: 2n2= O(n3) (c= 1, n0= 2)**

Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. EXAMPLE: 2n2= O(n3) (c= 1, n0= 2) functions, Not values Funny, “one-way” equality September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

6
**asymptotic upper bound**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

7
**Ω-notation (lower bounds)**

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

8
**Ω-notation (lower bounds)**

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). Ω(g(n))= { f(n) :there exist constants c> 0, n0> 0such that 0 ≤ cg(n) ≤ f(n) for all n≥n0} September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

9
**Ω-notation (lower bounds)**

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). Ω(g(n))= { f(n) :there exist constants c> 0, n0> 0such that 0 ≤ cg(n) ≤ f(n) for all n≥n0} EXAMPLE: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

10
**asymptotic lower bound**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

11
**-notation (tight bounds)**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

12
**-notation (tight bounds)**

EXAMPLE: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

13
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Graphic Example of Θ The definition of required every member of be asymptotically nonnegative. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

14
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Theorem 3.1. For any two functions f(n) and g(n), if and only if and September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

15
**ο-notation and ω-notation**

O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >. o(g(n))= { f(n) : for any constant c> 0 there is a constant n0> 0 such that 0 ≤ f(n) < cg(n) for all n≥ n0} EXAMPLE: 2n2= o(n3) (n0= 2/c) But 2n2 ≠ o(n2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

16
**ο-notation and ω-notation**

O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >. w(g(n))= { f(n) : for any constant c> 0 there is a constant n0> 0 such that 0 ≤ cg(n) < f(n) for all n≥ n0} EXAMPLE: n2/2 =w (n), but n2 /2 ≠ w(n2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

17
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Solving recurrences The analysis of merge sort from Lecture required us to solve a recurrence. Recurrences are like solving integrals, differential equations, etc. 。Learn a few tricks. Lecture 3: Applications of recurrences to divide-and-conquer algorithms. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

18
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Substitution method The most general method: 1.Guess the form of the solution. 2.Verify by induction. 3.Solve for constants. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

19
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Substitution method The most general method: 1.Guess the form of the solution. 2.Verify by induction. 3.Solve for constants. EXAMPLE: T(n) = 4T(n/2) + n • [Assume that T(1) = Θ(1).] • Guess O(n3). (Prove O and Ωseparately.) • Assume that T(k) ≤ ck3 for k< n . • Prove T(n) ≤cn3 by induction. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

20
**Example of substitution**

T(n) = 4T(n/2) + n ≤ 4c(n/2)3 + n = (c/2)n3 + n = cn3 –((c/2)n3 – n)← desired – residual ≤ cn3 ← desired Whenever (c/2)n3 – n ≥ 0, for example, if c ≥ 2 and n ≥ 1. residual September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

21
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Example (continued) •We must also handle the initial conditions, that is, ground the induction with base cases. •Base: T(n) = Θ(1) for all n< n0, where n0is a suitable constant. •For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if we pick c big enough. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

22
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Example (continued) •We must also handle the initial conditions, that is, ground the induction with base cases. •Base: T(n) = Θ(1) for all n< n0, where n0is a suitable constant. •For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if we pick c big enough. This bound is not tight! September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

23
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

24
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: T(n)= 4T(n/2) + n ≤ 4c(n/2)2 + n = cn2 + n = O(n2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

25
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: T(n)= 4T(n/2) + n ≤ 4c(n/2)2 + n = cn2 + n = O (n2) Wrong! We must prove the I.H. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

26
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: T(n)= 4T(n/2) + n ≤ 4c(n/2)2 + n = cn2 + n = O (n2) = cn2 – (- n) [ desired - residual] ≤ cn2 for no choice of c > 0. Lose! Wrong! We must prove the I.H. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

27
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

28
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n)= 4T(n/2) + n = 4(c1(n/2)2–c2(n/2))+ n = c1n2–2c2n+ n = c1n2–c2n–(c2n–n) ≤ c1n2–c2n if c2≥1. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

29
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n)= 4T(n/2) + n = 4(c1(n/2)2–c2(n/2))+ n = c1n2–2c2n+ n = c1n2–c2n–(c2n–n) ≤ c1n2–c2n if c2≥1. Pick c1 big enough to handle the initial conditions. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

30
**Recursion-tree method**

• A recursion tree models the costs (time) of a recursive execution of an algorithm. • The recursion-tree method can be unreliable, just like any method that uses ellipses (…). • The recursion-tree method promotes intuition, however. • The recursion tree method is good for generating guesses for the substitution method. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

31
**Example of recursion tree**

Solve T(n) = T(n/4) + T(n/2)+ n2: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

32
**Example of recursion tree**

Solve T(n) = T(n/4) + T(n/2)+ n2: T(n) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

33
**Example of recursion tree**

Solve T(n) = T(n/4) + T(n/2)+ n2: n2 T(n/4) T(n/2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

34
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

35
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

36
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

37
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

38
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

39
**Example of recursion tree**

geometric series September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

40
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

The master method The master method applies to recurrences of the form T(n) = aT(n/b) + f(n) , where a≥1, b> 1, and f is asymptotically positive. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

41
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Three common cases Compare f (n)with n logba: 1. f (n) = O (n logba–ε)for some constant ε> 0. • f (n) grows polynomially slower than n logba (by an nεfactor). Solution: T(n) = (n logba) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

42
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Three common cases Compare f (n)with n logba: 1. f (n) = O (n logba–ε)for some constant ε> 0. • f (n) grows polynomially slower than n logba (by an nεfactor). Solution: T(n) = (n logba) 2. f (n) = (n logbalgkn)for some constant k≥0. • f (n) and nl ogba grow at similar rates. Solution: T(n) = (n logbalgk+1n). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

43
**Three common cases (cont.)**

Compare f (n)with n logba: 3. f (n) = Ω(nlogba+ ε)for some constant ε> 0. • f (n) grows polynomially faster than n logba (by an nεfactor), and f (n) satisfies the regularity condition that af(n/b) ≤ cf(n) for some constant c< 1. Solution: T(n) = (f(n)). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

44
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX. T(n) = 4T(n/2) + n a =4, b= 2 ⇒n logba =n2; f(n) = n. CASE1: f(n) = O(n2–ε)for ε= 1. ∴T(n) = (n2). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

45
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX. T(n) = 4T(n/2) + n a =4, b= 2 ⇒n logba =n2; f(n) = n. CASE1: f(n) = O(n2–ε)for ε= 1. ∴T(n) = (n2). EX.T(n) = 4T(n/2) + n2 a =4, b= 2 ⇒ n logba =n2; f(n) = n2. CASE2: f(n) = (n2lg0n), that is, k = 0. ∴T(n) = (n2lgn). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

46
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX.T(n) = 4T(n/2) + n3 a =4, b= 2 ⇒n logba=n2; f(n) = n3. CASE3: f(n) = Ω(n2+ ε)for ε= 1 and 4(n/2)3≤cn3 (reg. cond.) for c= 1/2. ∴T(n) = (n3). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

47
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX.T(n) = 4T(n/2) + n3 a =4, b= 2 ⇒n logba=n2; f(n) = n3. CASE3: f(n) = Ω(n2+ ε)for ε= 1 and 4(n/2)3≤cn3 (reg. cond.) for c= 1/2. ∴T(n) = (n3). EX.T(n) = 4T(n/2) + n2/lgn a =4, b= 2 ⇒ n logba =n2; f(n) = n2/lgn. Master method does not apply. In particular, for every constant ε> 0, we have nε= ω(lgn). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

48
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

49
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

50
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

51
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

52
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem CASE1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

53
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: CASE2: (k= 0) The weight is approximately the same on each of the logbn levels. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

54
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem CASE3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

55
**Polynomials v.s. Exponentials**

A function is polynomial bounded if Exponentials: Any positive exponential function grows faster than any polynomial. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

56
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Logarithms A polylogarithmic function in n is a polynomial in the logarithm of n, A function f(n) is polylogarithmically bounded if for any constant a > 0. Any positive polynomial function grows faster than any polylogarithmic function. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

57
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Function iteration For example, if , then September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

58
**Slowly Growing function**

Since the number of atoms in the observable universe is estimated to be about , which is much less than , we rarely encounter a value of n such that September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

59
**Appendix: geometric series**

Return to last slide viewed. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google