Algorithms Growth of Functions

Some Notation NNatural numbers RReal numbers N + Positive natural numbers R + Positive real numbers R * Non-negative real numbers BBoolean constants {true, false}

Growth of Functions Asymptotic efficiency of algorithms – How does the running time of an algorithm increase with the size of the input in the limit as the input increases in size without bound? Asymptotic notation (“the order of”) – Define sets of functions that satisfy certain criteria and use these to characterize time and space complexity of algorithms

Big O Definition: For a given function g(n), O(g(n)) is the set of functions O(g(n))= {f(n): there exist positive constants c and n 0 such that 0  f(n)  c g(n) for all n  n 0 } c is the multiplicative constant n 0 is the threshold

Big O Big O is an upper bound on a function to within a constant function. O(g(n)) is a set of functions Commonly used notation f(n) = O(g(n)) Correct notation f(n)  O(g(n)) Meaningless statement O(g(n)) = f(n)

n c g(n) f(n) n0n0 f(n)  O(g(n))

Question: How do you demonstrate that f(n)  O(g(n)) Answer: Show that you can find values for c and n 0 such that 0  f(n)  c g(n) for all n  n 0

n c g(n) f(n) n0n0 f(n) may be negative or undefined for some values of n

Big O and Algorithms Principle of Invariance If some implementation of an algorithm never takes more than t(n) seconds to solve for an instance of size n, then any other implementations of the same algorithm take a time in the order of t(n) seconds. Therefore the algorithm takes time in the order of f(n) for any function f: N  R * such that t(n)  O(f(n))

Example Suppose t(n) = 20n n 2 -3n + 1  s Then t(n)  O(20n n 2 -3n + 1) since it is always the case that t(n)  O(t(n)) with c = 1 and n 0 = 0 But it is also the case that t(n)  O(20n 3 ) t(n)  O(n 3 )

Why not just use t(n)? Using simple functions for the order simplifies comparison of algorithms t(n) may be very difficult to determine exactly In general, we try to express the order of an algorithm’s running time using the simplest possible function f such that t(n)  O(f(n))

True or False? 3n + 4  O(n) True n/2  O(n) False (it is Upper bond should be n or greater n2) 100n n - 6  O(n) False n  O(20n) True n  O(n 2 ) True 6*2 n +n 2  O(n 2 ) False. 6*2 n +n 2  O(2 n ) True.

Common Terminology ComplexityTerm O(1)constant O(log n)logarithmic O(n)linear O(n lg n)n log n O(n b ) polynomial O(b n ) b > 1exponential O(n!)factorial

Big Omega Definition: For a given function g(n),  (g(n)) is the set of functions:  (g(n)) = {f(n): there exist positive constants c and n 0 such that 0  c g(n)  f(n) for all n  n 0 } Omega provides a lower bound for a function to within a constant factor

n c g(n) f(n) n0n0 f(n)   (g(n))

Alternative Definition f(n)   (g(n)) iff g(n)  O(f(n))

True or False? n  O(n) n  (n) n/2  O(n) n/2  (n) n + 1  O(n) n + 1  (n) n  O(1) n  (1) 3  O(n) 3  (n) n  O(n 2 ) n  (n 2 ) n 2  O(n) n 2  (n)

Big Theta Definition For a given function g(n),  (g(n)) is the set of functions:  (g(n)) = {f(n): there exist positive constants c 1, c 2 and n 0 such that 0  c 1 g(n)  f(n)  c 2 g(n) for all n  n 0 }

n c 1 g(n) f(n) n0n0 f(n)   (g(n)) c 2 g(n)

Alternative Definition  (g(n)) = O(g(n))  (g(n)) If we can find a simple function that gives both an upper and lower bound on the growth of the function, this is very useful. It is not always simple.

Little o Definition: For a given function g(n), o(g(n)) is the set of functions o(g(n))= {f(n): for any positive constant c, there exists a constant n 0 such that 0  f(n) < c g(n) for all n  n 0 } all values of c g(n) is greater then f(n) and these values are called little o. Denotes an upper bound that is not asymptotically tight Examples: 2n  o(n 2 ) but 2n 2  o(n 2 )

The function f(n) becomes insignificant relative to g(n) as n approaches infinity if the limit exists Alternative Definition for little o

little-omega Definition: For a given function g(n),  (g(n)) is the set of functions  (g(n))= {f(n): for any positive constant c, there exists a constant n 0 such that 0  c g(n) < f(n) for all n  n 0 } Denotes a lower bound that is not asymptotically tight Examples: n  (n 2 ) n  (sqrt(n)) n  (lg n)

Alternative Definition for little omega The function g(n) becomes insignificant relative to f(n) as n approaches infinity if the limit exists

Binary Relations Each of these notations can be viewed as a binary relation on a set of functions {t: N  R * } Properties of relations on a set transitivity: R is transitive iff for all a, b, c whenever a R b and b R c, then a R c reflexivity:R is reflexive iff for all a, a R a symmetry: R is symmetric iff for all a and b, a R b and b R a

Binary Relations cont. Property of two relations transpose symmetry: R 1 and R 2 exhibit transpose symmetry iff for all a and b when a R 1 b then b R 2 a

Transitivity f(n)  (g(n))  g(n)  (h(n))  f(n)  (h(n)) f(n)  (g(n))  g(n)  (h(n))  f(n)  (h(n)) f(n)  (g(n))  g(n)  (h(n))  f(n)  (h(n)) f(n)  (g(n))  g(n)  (h(n))  f(n)  (h(n)) f(n)  (g(n))  g(n)  (h(n))  f(n)  (h(n))

Reflexivity f(n)  (f(n)) f(n)  (f(n))) f(n)  (f(n))

Symmetry f(n)  (g(n))  g(n)  (f(n))) Transpose Symmetry f(n)  (g(n))  g(n)  (f(n))) f(n)  (g(n))  g(n)  (f(n)))

ORDER NOTATION Say that f isMean that f isWriteif small oh g slower than gf(n)  o(g(n)) big oh g no faster than gf(n)  O(g(n))  c, n o > o,  n  n o f(n)  c g(n) theta g about as fastf(n)   (g(n)) f(n)  O(g(n)) and as gg(n)  O(f(n)) omega g no slower than gf(n)   (g(n)) g(n)  O(f(n) little omega gfaster than gf(n)   (g(n))g(n)   (f(n))

Equivalence Relation An equivalence relation is a relation that is reflexive, symmetric and transitive. Which of the relations are equivalence relations? An equivalence relation partitions a set into equivalence classes. Describe the equivalence classes.