Big-O

Algorithm Analysis Exact analysis: produce a function f(n) measuring how many basic steps are needed for a given inputs n On any input of size n, the algorithm runs for at most 7n 2 + 3n + 8 steps Do we really need such a precise measurement? To compare two algorithm, we are concerned with how well the algorithm scales, Too much details of f(n) is meaningless! (asymptotic) analysis

Asymptotic Analysis Asymptotic analysis of an algorithm describes the relative efficiency of an algorithm when n gets very large

Asymptotic Analysis (cont’d) For large values of n, Algorithm 1 grows faster than Algorithm 2 N

Why does Complexity Matter? Assume that you have 3 algorithms to sort a list –f(n) = nlog 2 n –g(n) = n 2 –h(n) = n 3 Assume that each step takes 1 microsecond (i.e., seconds)

Asymptotic Analysis (cont’d)

Bound f(n) is contributed from N^2 The grow rate is bounded with such bound, we only need to refer to well known functions and do not need to be concerned with the details of the function Big-O notion: upper bound on the growth rate of a function, for sufficiently large values of n.

Big-O Notation Definition: f(n) = O(g(n)) if there exist positive constants c and N such that f(n)  cg(n) for all n  N Big-O expresses an upper bound on the growth rate of a function, for sufficiently large values of n. Graph to understand Big-O

An Example Prove that 2n = O(n 2 )

Proof with Induction

Typical functions

Properties of Big-O Notation Fact 1 (Transitivity) –If f(n)=O(g(n)) and g(n)=O(h(n)), then f(n)=O(h(n)) –Prove it (done in class) Fact 2 –If f(n)=O(h(n)) and g(n)=O(h(n)), then f(n)+g(n)=O(h(n)) Fact 3 –The function an k =O(n k ) Fact 4 –The function n k =O(n k+j ) for any positive j

Properties of Big-O Notation (cont’d) It follows from those facts that every polynomial is big-O of n raised to the largest power f(n) = a k n k + a k-1 n k a 1 n 1 + a 0 = O(n k )

Properties of Big-O Notation (cont’d) Fact 5 –If f(n)=cg(n), then f(n)=O(g(n)) Fact 6 –The function log a n = O(log b n) for any positive numbers a and b  1 –WHY? Fact 7 –log a n = O(lgn) for any positive a  1, where lg n = log 2 n