COP 3530 Spring2012 Data Structures & Algorithms Discussion Session Week 5

Outline Growth of functions – Big Oh – Omega – Theta – Little Oh

Growth of Functions Gives a simple view of the algorithm’s efficiency. Allows us to compare the relative performance of alternative algorithms. f1(n) is O(n) f2(n) is O(n^2)

Growth of Functions Exact running time of an algorithm is usually hard to compute, and it’s unnecessary. For large enough inputs, the lower-order terms of an exact running time are dominated by high-order terms. f(n) = n^2 + 5n n^2 >> 5n + 234, when n is large enough

Asymptotic Notation: Big Oh (O) f(n)= O(g(n)) iff there exist positive constants c and n0 such that f(n) ≤ cg(n) for all n ≥ n0 O-notation to give an upper bound on a function

Asymptotic Notation: Big Oh (O) Example 1[linear function] f(n) = 3n+2 For n >= 2, 3n+2 <= 3n+n <= 4n. So f(n) = O(n). We can arrive the same conclusion in other ways. For example, 3n The specific values of c and n0 used to satisfy the definition of big oh are not important, we only say f(n) is big oh of g(n). c/n0 do not matter.

Asymptotic Notation: Big Oh (O) Example 2[quadric function] f(n) = 10n^2+4n+2 For n >= 2, f(n) <= 10n^2+5n. For n>= 5, 5n < n^2. Hence for n>= n0 = 5, f(n) <= 10n^2+n^2 = 11n^2. Therefore, f(n) = O(n^2). Only the highest order term matters !!!

Asymptotic Notation: Big Oh (O) Example 3[exponential function] f(n) = 6*2^n + n^2 For n>=4, n^2 <= 2^n. f(n) =4 Therefore, 6*2^n + n^2 = O(2^n) Example 4[constant function] f(n) = 3 For any n, f(n) <= 4*1 Therefore, f(n) = O(1)

Asymptotic Notation: Big Oh (O) Example 5[loose bounds] f(n) = 3n+3 For n >= 10, 3n+3 <= 3n^2. Therefore, f(n) = O(n^2). Usually, we mean tight upper bound when using big oh notation. Example 6[Incorrect bounds] 3n+2 != O(1) since we cannot find n0/c such that 3n + 2 =c0 (n can be infinity). Similarly, 10n^2 + 6n + 2 != O(n).

Asymptotic Notation: Big Oh (O) The specific values of c and n0 used to satisfy the definition of big oh are not important, we only say f(n) is big oh of g(n). c/n0 do NOT matter, WHY? f(n)=n. It’s close to 10n when comparing with n^2, n^3 f(n) is relatively small when n<n0

Asymptotic Notation: Omega Notation Big oh provides an asymptotic upper bound on a function. Omega provides an asymptotic lower bound on a function.

Asymptotic Notation: Omega Notation Example 7 f(n) = 3n+3 > 3n for all n. So f(n) = Omega(n) Example 8[loose bounds] f(n) = 3n+3 > 1 for all n, so f(n) = Omega(1)

Asymptotic Notation: Theta Notation Theta notation is used when function f can be bounded both from above and below by the same function g

Asymptotic Notation: Theta Notation Example 9: f(n) = 3n+3 is Theta(n), since n = 3. Similarly, f(n) = 3n+2 is Theta(n) f(n) = 5n^2 - 10n + 9 is Theta(n^2) f(n) is Theta(g(n)) iff f(n) is Omega(g(n)) and O(g(n))

Asymptotic Notation: Little oh (o) The asymptotic upper bound provided by O-notation may or may not be asymptotically tight. 2n = O(n) is tight, 2n = O(n^2) is not tight. We use o-notation to denote an upper bound that is NOT asymptotically tight.

Asymptotic Notation: Little oh (o) f(n) = o(g(n)) iff f(n) = O(g(n)) and f(n) != Omega(g(n)) Example 10 3n+2 = o(n^2) as 3n+2 = O(n^2) and 3n+2 != Omega(n^2) Example 11 3n+2 != o(n) as 3n+2 = Omega(n)

Review Big oh: upper bound on a function. Omega: lower bound. Theta: lower and upper bound. - f(n) is Theta(g(n)) iff f(n) is O(g(n)) and Omega(g(n)) Little oh: loose upper bound. - f(n) = o(g(n)) iff f(n) = O(g(n)) and f(n) != Omega(g(n))

Office Hour This Week: Thursday 9 th period at E309