Download presentation

Presentation is loading. Please wait.

1
**Introduction to Algorithms**

LECTURE2 Asymptotic Notation •O-, Ω-, and Θ-notation Recurrences •Substitution method •Iterating the recurrence •Recursion tree •Master method September 12, 2005 Copyright© Erik D. Demaine and Charles E. Leiserson

2
**Asymptotic notation O-notation (upper bounds):**

We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

3
**EXAMPLE: 2n2= O(n3) (c= 1, n0= 2)**

Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. EXAMPLE: 2n2= O(n3) (c= 1, n0= 2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

4
**EXAMPLE: 2n2= O(n3) (c= 1, n0= 2)**

Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. EXAMPLE: 2n2= O(n3) (c= 1, n0= 2) functions, Not values September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

5
**EXAMPLE: 2n2= O(n3) (c= 1, n0= 2)**

Asymptotic notation O-notation (upper bounds): We write f(n) = O(g(n)) if there Exist constants c>0, n0>0 such That 0 ≤f(n) ≤cg(n) for all n≥n0. EXAMPLE: 2n2= O(n3) (c= 1, n0= 2) functions, Not values Funny, “one-way” equality September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

6
**asymptotic upper bound**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

7
**Ω-notation (lower bounds)**

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

8
**Ω-notation (lower bounds)**

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). Ω(g(n))= { f(n) :there exist constants c> 0, n0> 0such that 0 ≤ cg(n) ≤ f(n) for all n≥n0} September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

9
**Ω-notation (lower bounds)**

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). Ω(g(n))= { f(n) :there exist constants c> 0, n0> 0such that 0 ≤ cg(n) ≤ f(n) for all n≥n0} EXAMPLE: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

10
**asymptotic lower bound**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

11
**-notation (tight bounds)**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

12
**-notation (tight bounds)**

EXAMPLE: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

13
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Graphic Example of Θ The definition of required every member of be asymptotically nonnegative. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

14
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Theorem 3.1. For any two functions f(n) and g(n), if and only if and September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

15
**ο-notation and ω-notation**

O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >. o(g(n))= { f(n) : for any constant c> 0 there is a constant n0> 0 such that 0 ≤ f(n) < cg(n) for all n≥ n0} EXAMPLE: 2n2= o(n3) (n0= 2/c) But 2n2 ≠ o(n2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

16
**ο-notation and ω-notation**

O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >. w(g(n))= { f(n) : for any constant c> 0 there is a constant n0> 0 such that 0 ≤ cg(n) < f(n) for all n≥ n0} EXAMPLE: n2/2 =w (n), but n2 /2 ≠ w(n2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

17
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Solving recurrences The analysis of merge sort from Lecture required us to solve a recurrence. Recurrences are like solving integrals, differential equations, etc. 。Learn a few tricks. Lecture 3: Applications of recurrences to divide-and-conquer algorithms. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

18
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Substitution method The most general method: 1.Guess the form of the solution. 2.Verify by induction. 3.Solve for constants. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

19
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Substitution method The most general method: 1.Guess the form of the solution. 2.Verify by induction. 3.Solve for constants. EXAMPLE: T(n) = 4T(n/2) + n • [Assume that T(1) = Θ(1).] • Guess O(n3). (Prove O and Ωseparately.) • Assume that T(k) ≤ ck3 for k< n . • Prove T(n) ≤cn3 by induction. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

20
**Example of substitution**

T(n) = 4T(n/2) + n ≤ 4c(n/2)3 + n = (c/2)n3 + n = cn3 –((c/2)n3 – n)← desired – residual ≤ cn3 ← desired Whenever (c/2)n3 – n ≥ 0, for example, if c ≥ 2 and n ≥ 1. residual September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

21
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Example (continued) •We must also handle the initial conditions, that is, ground the induction with base cases. •Base: T(n) = Θ(1) for all n< n0, where n0is a suitable constant. •For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if we pick c big enough. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

22
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Example (continued) •We must also handle the initial conditions, that is, ground the induction with base cases. •Base: T(n) = Θ(1) for all n< n0, where n0is a suitable constant. •For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if we pick c big enough. This bound is not tight! September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

23
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

24
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: T(n)= 4T(n/2) + n ≤ 4c(n/2)2 + n = cn2 + n = O(n2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

25
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: T(n)= 4T(n/2) + n ≤ 4c(n/2)2 + n = cn2 + n = O (n2) Wrong! We must prove the I.H. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

26
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound? We shall prove that T(n) = O(n2). Assume that T(k) ≤ ck2 for k < n: T(n)= 4T(n/2) + n ≤ 4c(n/2)2 + n = cn2 + n = O (n2) = cn2 – (- n) [ desired - residual] ≤ cn2 for no choice of c > 0. Lose! Wrong! We must prove the I.H. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

27
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

28
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n)= 4T(n/2) + n = 4(c1(n/2)2–c2(n/2))+ n = c1n2–2c2n+ n = c1n2–c2n–(c2n–n) ≤ c1n2–c2n if c2≥1. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

29
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

A tighter upper bound! IDEA: Strengthen the inductive hypothesis. • Subtract a low-order term. Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n)= 4T(n/2) + n = 4(c1(n/2)2–c2(n/2))+ n = c1n2–2c2n+ n = c1n2–c2n–(c2n–n) ≤ c1n2–c2n if c2≥1. Pick c1 big enough to handle the initial conditions. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

30
**Recursion-tree method**

• A recursion tree models the costs (time) of a recursive execution of an algorithm. • The recursion-tree method can be unreliable, just like any method that uses ellipses (…). • The recursion-tree method promotes intuition, however. • The recursion tree method is good for generating guesses for the substitution method. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

31
**Example of recursion tree**

Solve T(n) = T(n/4) + T(n/2)+ n2: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

32
**Example of recursion tree**

Solve T(n) = T(n/4) + T(n/2)+ n2: T(n) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

33
**Example of recursion tree**

Solve T(n) = T(n/4) + T(n/2)+ n2: n2 T(n/4) T(n/2) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

34
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

35
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

36
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

37
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

38
**Example of recursion tree**

September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

39
**Example of recursion tree**

geometric series September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

40
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

The master method The master method applies to recurrences of the form T(n) = aT(n/b) + f(n) , where a≥1, b> 1, and f is asymptotically positive. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

41
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Three common cases Compare f (n)with n logba: 1. f (n) = O (n logba–ε)for some constant ε> 0. • f (n) grows polynomially slower than n logba (by an nεfactor). Solution: T(n) = (n logba) September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

42
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Three common cases Compare f (n)with n logba: 1. f (n) = O (n logba–ε)for some constant ε> 0. • f (n) grows polynomially slower than n logba (by an nεfactor). Solution: T(n) = (n logba) 2. f (n) = (n logbalgkn)for some constant k≥0. • f (n) and nl ogba grow at similar rates. Solution: T(n) = (n logbalgk+1n). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

43
**Three common cases (cont.)**

Compare f (n)with n logba: 3. f (n) = Ω(nlogba+ ε)for some constant ε> 0. • f (n) grows polynomially faster than n logba (by an nεfactor), and f (n) satisfies the regularity condition that af(n/b) ≤ cf(n) for some constant c< 1. Solution: T(n) = (f(n)). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

44
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX. T(n) = 4T(n/2) + n a =4, b= 2 ⇒n logba =n2; f(n) = n. CASE1: f(n) = O(n2–ε)for ε= 1. ∴T(n) = (n2). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

45
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX. T(n) = 4T(n/2) + n a =4, b= 2 ⇒n logba =n2; f(n) = n. CASE1: f(n) = O(n2–ε)for ε= 1. ∴T(n) = (n2). EX.T(n) = 4T(n/2) + n2 a =4, b= 2 ⇒ n logba =n2; f(n) = n2. CASE2: f(n) = (n2lg0n), that is, k = 0. ∴T(n) = (n2lgn). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

46
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX.T(n) = 4T(n/2) + n3 a =4, b= 2 ⇒n logba=n2; f(n) = n3. CASE3: f(n) = Ω(n2+ ε)for ε= 1 and 4(n/2)3≤cn3 (reg. cond.) for c= 1/2. ∴T(n) = (n3). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

47
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Examples EX.T(n) = 4T(n/2) + n3 a =4, b= 2 ⇒n logba=n2; f(n) = n3. CASE3: f(n) = Ω(n2+ ε)for ε= 1 and 4(n/2)3≤cn3 (reg. cond.) for c= 1/2. ∴T(n) = (n3). EX.T(n) = 4T(n/2) + n2/lgn a =4, b= 2 ⇒ n logba =n2; f(n) = n2/lgn. Master method does not apply. In particular, for every constant ε> 0, we have nε= ω(lgn). September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

48
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

49
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

50
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

51
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

52
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem CASE1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

53
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem Recursion tree: CASE2: (k= 0) The weight is approximately the same on each of the logbn levels. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

54
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Idea of master theorem CASE3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. Recursion tree: September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

55
**Polynomials v.s. Exponentials**

A function is polynomial bounded if Exponentials: Any positive exponential function grows faster than any polynomial. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

56
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Logarithms A polylogarithmic function in n is a polynomial in the logarithm of n, A function f(n) is polylogarithmically bounded if for any constant a > 0. Any positive polynomial function grows faster than any polylogarithmic function. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

57
**Copyright ?2001-5 Erik D. Demaine and Charles E. Leiserson**

Function iteration For example, if , then September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

58
**Slowly Growing function**

Since the number of atoms in the observable universe is estimated to be about , which is much less than , we rarely encounter a value of n such that September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

59
**Appendix: geometric series**

Return to last slide viewed. September 12, 2005 Copyright ? Erik D. Demaine and Charles E. Leiserson

Similar presentations

Presentation is loading. Please wait....

OK

Essential Cell Biology

Essential Cell Biology

© 2018 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google